2023-02-13 12:23:25,294 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-13 12:23:25,484 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d 2023-02-13 12:23:25,495 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 12:23:25,495 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71, deleteOnExit=true 2023-02-13 12:23:25,496 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 12:23:25,496 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/test.cache.data in system properties and HBase conf 2023-02-13 12:23:25,496 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 12:23:25,497 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/hadoop.log.dir in system properties and HBase conf 2023-02-13 12:23:25,497 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 12:23:25,498 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 12:23:25,498 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 12:23:25,599 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 12:23:25,912 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 12:23:25,916 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 12:23:25,916 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 12:23:25,916 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 12:23:25,916 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 12:23:25,917 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 12:23:25,917 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 12:23:25,917 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 12:23:25,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 12:23:25,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 12:23:25,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/nfs.dump.dir in system properties and HBase conf 2023-02-13 12:23:25,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/java.io.tmpdir in system properties and HBase conf 2023-02-13 12:23:25,919 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 12:23:25,919 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 12:23:25,919 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 12:23:26,320 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 12:23:26,323 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 12:23:26,692 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 12:23:26,840 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 12:23:26,852 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:23:26,883 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:23:26,910 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/java.io.tmpdir/Jetty_localhost_localdomain_35949_hdfs____9emldi/webapp 2023-02-13 12:23:27,011 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:35949 2023-02-13 12:23:27,043 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 12:23:27,044 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 12:23:27,771 WARN [Listener at localhost.localdomain/39971] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:23:27,898 WARN [Listener at localhost.localdomain/39971] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 12:23:27,920 WARN [Listener at localhost.localdomain/39971] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 12:23:27,927 INFO [Listener at localhost.localdomain/39971] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 12:23:27,934 INFO [Listener at localhost.localdomain/39971] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/java.io.tmpdir/Jetty_localhost_37003_datanode____.bvcoju/webapp 2023-02-13 12:23:28,053 INFO [Listener at localhost.localdomain/39971] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37003 2023-02-13 12:23:28,438 WARN [Listener at localhost.localdomain/33629] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 12:23:29,257 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc33e8cbc90505d58: Processing first storage report for DS-c89849b1-6287-4773-82f7-d8ae211d07dc from datanode 368b0532-ef68-4161-bbce-2cda1773f909 2023-02-13 12:23:29,258 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc33e8cbc90505d58: from storage DS-c89849b1-6287-4773-82f7-d8ae211d07dc node DatanodeRegistration(127.0.0.1:46225, datanodeUuid=368b0532-ef68-4161-bbce-2cda1773f909, infoPort=34071, infoSecurePort=0, ipcPort=33629, storageInfo=lv=-57;cid=testClusterID;nsid=1609337573;c=1676291006386), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 12:23:29,259 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc33e8cbc90505d58: Processing first storage report for DS-d4cfb79e-4513-4f2b-8cd6-a0314df369fe from datanode 368b0532-ef68-4161-bbce-2cda1773f909 2023-02-13 12:23:29,259 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc33e8cbc90505d58: from storage DS-d4cfb79e-4513-4f2b-8cd6-a0314df369fe node DatanodeRegistration(127.0.0.1:46225, datanodeUuid=368b0532-ef68-4161-bbce-2cda1773f909, infoPort=34071, infoSecurePort=0, ipcPort=33629, storageInfo=lv=-57;cid=testClusterID;nsid=1609337573;c=1676291006386), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 12:23:29,312 DEBUG [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d 2023-02-13 12:23:29,361 INFO [Listener at localhost.localdomain/33629] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/zookeeper_0, clientPort=64508, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 12:23:29,375 INFO [Listener at localhost.localdomain/33629] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64508 2023-02-13 12:23:29,385 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:29,387 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:30,112 INFO [Listener at localhost.localdomain/33629] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0 with version=8 2023-02-13 12:23:30,113 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/hbase-staging 2023-02-13 12:23:30,355 INFO [Listener at localhost.localdomain/33629] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 12:23:30,714 INFO [Listener at localhost.localdomain/33629] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=3 2023-02-13 12:23:30,739 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:30,739 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:30,740 INFO [Listener at localhost.localdomain/33629] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:23:30,740 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:30,740 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:23:30,852 INFO [Listener at localhost.localdomain/33629] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:23:30,911 DEBUG [Listener at localhost.localdomain/33629] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 12:23:30,985 INFO [Listener at localhost.localdomain/33629] ipc.NettyRpcServer(120): Bind to /188.40.62.62:41493 2023-02-13 12:23:30,995 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:30,997 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:31,014 INFO [Listener at localhost.localdomain/33629] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41493 connecting to ZooKeeper ensemble=127.0.0.1:64508 2023-02-13 12:23:31,124 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:414930x0, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:23:31,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:41493-0x10052d859b90000 connected 2023-02-13 12:23:31,211 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:23:31,213 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:23:31,218 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:23:31,229 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41493 2023-02-13 12:23:31,230 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41493 2023-02-13 12:23:31,230 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41493 2023-02-13 12:23:31,230 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41493 2023-02-13 12:23:31,231 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41493 2023-02-13 12:23:31,236 INFO [Listener at localhost.localdomain/33629] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0, hbase.cluster.distributed=false 2023-02-13 12:23:31,294 INFO [Listener at localhost.localdomain/33629] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=3 2023-02-13 12:23:31,294 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:31,294 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:31,294 INFO [Listener at localhost.localdomain/33629] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 12:23:31,295 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 12:23:31,295 INFO [Listener at localhost.localdomain/33629] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 12:23:31,299 INFO [Listener at localhost.localdomain/33629] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 12:23:31,302 INFO [Listener at localhost.localdomain/33629] ipc.NettyRpcServer(120): Bind to /188.40.62.62:38471 2023-02-13 12:23:31,304 INFO [Listener at localhost.localdomain/33629] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 12:23:31,309 DEBUG [Listener at localhost.localdomain/33629] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 12:23:31,309 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:31,312 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:31,313 INFO [Listener at localhost.localdomain/33629] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38471 connecting to ZooKeeper ensemble=127.0.0.1:64508 2023-02-13 12:23:31,324 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:384710x0, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 12:23:31,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38471-0x10052d859b90001 connected 2023-02-13 12:23:31,325 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:23:31,327 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:23:31,327 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 12:23:31,328 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38471 2023-02-13 12:23:31,328 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38471 2023-02-13 12:23:31,329 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38471 2023-02-13 12:23:31,329 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38471 2023-02-13 12:23:31,329 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38471 2023-02-13 12:23:31,331 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:31,350 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 12:23:31,352 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:31,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:23:31,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 12:23:31,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:31,379 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 12:23:31,381 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,41493,1676291010224 from backup master directory 2023-02-13 12:23:31,381 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 12:23:31,391 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:31,391 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 12:23:31,391 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:23:31,392 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:31,394 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 12:23:31,395 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 12:23:31,879 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/hbase.id with ID: 645b0535-398e-4042-82e3-852f571d04aa 2023-02-13 12:23:31,924 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:31,974 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:32,016 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x241e3d53 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:32,054 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@55b062bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:32,072 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 12:23:32,073 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 12:23:32,087 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 12:23:32,088 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 12:23:32,089 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:23:32,092 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:23:32,093 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:23:32,120 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store-tmp 2023-02-13 12:23:32,551 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:32,552 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 12:23:32,552 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:23:32,552 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:23:32,552 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 12:23:32,552 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:23:32,553 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:23:32,553 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676291012551Disabling compacts and flushes for region at 1676291012552 (+1 ms)Disabling writes for close at 1676291012552Writing region close event to WAL at 1676291012553 (+1 ms)Closed at 1676291012553 2023-02-13 12:23:32,555 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/WALs/jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:32,580 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C41493%2C1676291010224, suffix=, logDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/WALs/jenkins-hbase16.apache.org,41493,1676291010224, archiveDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/oldWALs, maxLogs=10 2023-02-13 12:23:32,627 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK] 2023-02-13 12:23:32,634 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 12:23:32,689 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/WALs/jenkins-hbase16.apache.org,41493,1676291010224/jenkins-hbase16.apache.org%2C41493%2C1676291010224.1676291012588 2023-02-13 12:23:32,690 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK]] 2023-02-13 12:23:32,691 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:32,691 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:32,695 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,696 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,753 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,759 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 12:23:32,781 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 12:23:32,794 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:32,799 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,801 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,816 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 12:23:32,821 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:23:32,822 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64415206, jitterRate=-0.04013863205909729}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:23:32,822 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676291012725Initializing all the Stores at 1676291012727 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676291012728 (+1 ms)Cleaning up temporary data from old regions at 1676291012807 (+79 ms)Cleaning up detritus from prior splits at 1676291012808 (+1 ms)Region opened successfully at 1676291012822 (+14 ms) 2023-02-13 12:23:32,824 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 12:23:32,842 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-13 12:23:32,843 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-13 12:23:32,845 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 12:23:32,846 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 12:23:32,872 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 25 msec 2023-02-13 12:23:32,872 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 12:23:32,894 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 12:23:32,899 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 12:23:32,923 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 12:23:32,926 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 12:23:32,928 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 12:23:32,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 12:23:32,936 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 12:23:32,974 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:32,976 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 12:23:32,976 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 12:23:32,992 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 12:23:33,003 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:23:33,003 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 12:23:33,003 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:33,004 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase16.apache.org,41493,1676291010224, sessionid=0x10052d859b90000, setting cluster-up flag (Was=false) 2023-02-13 12:23:33,032 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:33,058 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 12:23:33,059 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:33,078 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:33,103 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 12:23:33,105 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:33,107 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp 2023-02-13 12:23:33,134 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(952): ClusterId : 645b0535-398e-4042-82e3-852f571d04aa 2023-02-13 12:23:33,138 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 12:23:33,151 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 12:23:33,151 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 12:23:33,163 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 12:23:33,164 DEBUG [RS:0;jenkins-hbase16:38471] zookeeper.ReadOnlyZKClient(139): Connect 0x68aa53fc to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:33,175 DEBUG [RS:0;jenkins-hbase16:38471] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@666db099, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:33,175 DEBUG [RS:0;jenkins-hbase16:38471] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6252388c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:23:33,193 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:38471 2023-02-13 12:23:33,196 INFO [RS:0;jenkins-hbase16:38471] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 12:23:33,196 INFO [RS:0;jenkins-hbase16:38471] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 12:23:33,196 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 12:23:33,198 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,41493,1676291010224 with isa=jenkins-hbase16.apache.org/188.40.62.62:38471, startcode=1676291011293 2023-02-13 12:23:33,204 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 12:23:33,212 DEBUG [RS:0;jenkins-hbase16:38471] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 12:23:33,212 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:23:33,212 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:23:33,212 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:23:33,212 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 12:23:33,213 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-02-13 12:23:33,213 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,213 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:23:33,213 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,214 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676291043214 2023-02-13 12:23:33,216 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 12:23:33,220 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 12:23:33,221 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 12:23:33,225 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 12:23:33,227 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 12:23:33,233 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 12:23:33,233 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 12:23:33,234 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 12:23:33,234 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 12:23:33,235 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,241 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 12:23:33,243 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 12:23:33,243 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 12:23:33,246 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 12:23:33,247 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 12:23:33,249 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676291013249,5,FailOnTimeoutGroup] 2023-02-13 12:23:33,250 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676291013250,5,FailOnTimeoutGroup] 2023-02-13 12:23:33,251 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,251 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 12:23:33,253 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,254 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,267 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:47517, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 12:23:33,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,290 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0 2023-02-13 12:23:33,290 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:39971 2023-02-13 12:23:33,290 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 12:23:33,303 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:23:33,304 DEBUG [RS:0;jenkins-hbase16:38471] zookeeper.ZKUtil(162): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,304 WARN [RS:0;jenkins-hbase16:38471] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 12:23:33,305 INFO [RS:0;jenkins-hbase16:38471] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:23:33,305 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/WALs/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,306 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,38471,1676291011293] 2023-02-13 12:23:33,317 DEBUG [RS:0;jenkins-hbase16:38471] zookeeper.ZKUtil(162): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,328 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 12:23:33,335 INFO [RS:0;jenkins-hbase16:38471] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 12:23:33,351 INFO [RS:0;jenkins-hbase16:38471] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 12:23:33,354 INFO [RS:0;jenkins-hbase16:38471] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:23:33,354 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,355 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 12:23:33,360 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,361 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,361 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,361 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,361 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,361 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,362 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 12:23:33,362 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,362 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,362 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,362 DEBUG [RS:0;jenkins-hbase16:38471] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 12:23:33,363 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,363 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,363 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,376 INFO [RS:0;jenkins-hbase16:38471] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 12:23:33,378 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,38471,1676291011293-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:33,391 INFO [RS:0;jenkins-hbase16:38471] regionserver.Replication(203): jenkins-hbase16.apache.org,38471,1676291011293 started 2023-02-13 12:23:33,391 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,38471,1676291011293, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:38471, sessionid=0x10052d859b90001 2023-02-13 12:23:33,391 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 12:23:33,391 DEBUG [RS:0;jenkins-hbase16:38471] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,392 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,38471,1676291011293' 2023-02-13 12:23:33,392 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:33,392 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:33,393 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 12:23:33,393 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 12:23:33,393 DEBUG [RS:0;jenkins-hbase16:38471] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:33,393 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,38471,1676291011293' 2023-02-13 12:23:33,393 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:23:33,394 DEBUG [RS:0;jenkins-hbase16:38471] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:23:33,394 DEBUG [RS:0;jenkins-hbase16:38471] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 12:23:33,394 INFO [RS:0;jenkins-hbase16:38471] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 12:23:33,394 INFO [RS:0;jenkins-hbase16:38471] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 12:23:33,510 INFO [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C38471%2C1676291011293, suffix=, logDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/WALs/jenkins-hbase16.apache.org,38471,1676291011293, archiveDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/oldWALs, maxLogs=32 2023-02-13 12:23:33,527 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK] 2023-02-13 12:23:33,531 INFO [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/WALs/jenkins-hbase16.apache.org,38471,1676291011293/jenkins-hbase16.apache.org%2C38471%2C1676291011293.1676291013513 2023-02-13 12:23:33,532 DEBUG [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK]] 2023-02-13 12:23:33,658 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:33,659 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:33,659 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0 2023-02-13 12:23:34,074 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:34,077 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 12:23:34,081 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/info 2023-02-13 12:23:34,082 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 12:23:34,084 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,084 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 12:23:34,087 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:23:34,088 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 12:23:34,088 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,089 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 12:23:34,091 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/table 2023-02-13 12:23:34,091 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 12:23:34,092 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,094 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740 2023-02-13 12:23:34,095 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740 2023-02-13 12:23:34,099 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 12:23:34,102 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 12:23:34,106 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:23:34,107 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64861276, jitterRate=-0.033491671085357666}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 12:23:34,107 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676291014074Initializing all the Stores at 1676291014076 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676291014077 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676291014077Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676291014077Cleaning up temporary data from old regions at 1676291014097 (+20 ms)Cleaning up detritus from prior splits at 1676291014098 (+1 ms)Region opened successfully at 1676291014107 (+9 ms) 2023-02-13 12:23:34,107 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 12:23:34,107 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 12:23:34,107 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 12:23:34,108 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 12:23:34,108 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 12:23:34,108 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 12:23:34,109 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676291014107Disabling compacts and flushes for region at 1676291014107Disabling writes for close at 1676291014108 (+1 ms)Writing region close event to WAL at 1676291014108Closed at 1676291014108 2023-02-13 12:23:34,114 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 12:23:34,114 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 12:23:34,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 12:23:34,134 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 12:23:34,136 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 12:23:34,288 DEBUG [jenkins-hbase16:41493] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-13 12:23:34,292 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,38471,1676291011293, state=OPENING 2023-02-13 12:23:34,307 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 12:23:34,316 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:34,317 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 12:23:34,320 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:34,510 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:34,511 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:23:34,514 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43748, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:23:34,524 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 12:23:34,524 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 12:23:34,528 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C38471%2C1676291011293.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/WALs/jenkins-hbase16.apache.org,38471,1676291011293, archiveDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/oldWALs, maxLogs=32 2023-02-13 12:23:34,544 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK] 2023-02-13 12:23:34,549 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/WALs/jenkins-hbase16.apache.org,38471,1676291011293/jenkins-hbase16.apache.org%2C38471%2C1676291011293.meta.1676291014530.meta 2023-02-13 12:23:34,549 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46225,DS-c89849b1-6287-4773-82f7-d8ae211d07dc,DISK]] 2023-02-13 12:23:34,550 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:34,552 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 12:23:34,567 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 12:23:34,571 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 12:23:34,575 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 12:23:34,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:34,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 12:23:34,576 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 12:23:34,578 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 12:23:34,580 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/info 2023-02-13 12:23:34,580 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/info 2023-02-13 12:23:34,581 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 12:23:34,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 12:23:34,583 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:23:34,583 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/rep_barrier 2023-02-13 12:23:34,584 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 12:23:34,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,585 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 12:23:34,587 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/table 2023-02-13 12:23:34,587 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/table 2023-02-13 12:23:34,587 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 12:23:34,588 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:34,591 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740 2023-02-13 12:23:34,594 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740 2023-02-13 12:23:34,597 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 12:23:34,599 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 12:23:34,601 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62575203, jitterRate=-0.06755681335926056}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 12:23:34,601 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676291014576Writing region info on filesystem at 1676291014576Initializing all the Stores at 1676291014578 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676291014578Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676291014578Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676291014578Cleaning up temporary data from old regions at 1676291014596 (+18 ms)Cleaning up detritus from prior splits at 1676291014596Running coprocessor post-open hooks at 1676291014601 (+5 ms)Region opened successfully at 1676291014601 2023-02-13 12:23:34,610 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676291014504 2023-02-13 12:23:34,626 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 12:23:34,627 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 12:23:34,627 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,38471,1676291011293, state=OPEN 2023-02-13 12:23:34,636 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 12:23:34,637 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 12:23:34,642 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 12:23:34,642 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,38471,1676291011293 in 316 msec 2023-02-13 12:23:34,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 12:23:34,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 522 msec 2023-02-13 12:23:34,656 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5050 sec 2023-02-13 12:23:34,657 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676291011369, completionTime=-1 2023-02-13 12:23:34,657 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-13 12:23:34,657 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 12:23:34,712 DEBUG [hconnection-0x7a72179a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:23:34,717 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43762, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:23:34,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-13 12:23:34,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676291074737 2023-02-13 12:23:34,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676291134737 2023-02-13 12:23:34,737 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 79 msec 2023-02-13 12:23:34,780 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41493,1676291010224-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:34,781 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41493,1676291010224-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:34,781 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41493,1676291010224-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:34,782 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:41493, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:34,782 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 12:23:34,792 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 12:23:34,797 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 12:23:34,798 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 12:23:34,806 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 12:23:34,811 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:23:34,813 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:23:34,835 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:34,837 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7 empty. 2023-02-13 12:23:34,838 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:34,838 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 12:23:35,270 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:35,272 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => e7ec1481802597d098b27d5a31dd86f7, NAME => 'hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:23:35,284 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:35,284 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing e7ec1481802597d098b27d5a31dd86f7, disabling compactions & flushes 2023-02-13 12:23:35,284 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,284 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,284 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. after waiting 0 ms 2023-02-13 12:23:35,285 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,285 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,285 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for e7ec1481802597d098b27d5a31dd86f7: Waiting for close lock at 1676291015284Disabling compacts and flushes for region at 1676291015284Disabling writes for close at 1676291015284Writing region close event to WAL at 1676291015285 (+1 ms)Closed at 1676291015285 2023-02-13 12:23:35,289 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:23:35,302 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676291015292"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291015292"}]},"ts":"1676291015292"} 2023-02-13 12:23:35,326 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:23:35,327 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:23:35,332 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291015328"}]},"ts":"1676291015328"} 2023-02-13 12:23:35,336 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 12:23:35,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7ec1481802597d098b27d5a31dd86f7, ASSIGN}] 2023-02-13 12:23:35,389 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e7ec1481802597d098b27d5a31dd86f7, ASSIGN 2023-02-13 12:23:35,390 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e7ec1481802597d098b27d5a31dd86f7, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:23:35,543 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e7ec1481802597d098b27d5a31dd86f7, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:35,544 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676291015542"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291015542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291015542"}]},"ts":"1676291015542"} 2023-02-13 12:23:35,550 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e7ec1481802597d098b27d5a31dd86f7, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:35,710 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,711 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e7ec1481802597d098b27d5a31dd86f7, NAME => 'hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:35,712 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,713 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:35,713 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,713 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,715 INFO [StoreOpener-e7ec1481802597d098b27d5a31dd86f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,717 DEBUG [StoreOpener-e7ec1481802597d098b27d5a31dd86f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/info 2023-02-13 12:23:35,717 DEBUG [StoreOpener-e7ec1481802597d098b27d5a31dd86f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/info 2023-02-13 12:23:35,717 INFO [StoreOpener-e7ec1481802597d098b27d5a31dd86f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e7ec1481802597d098b27d5a31dd86f7 columnFamilyName info 2023-02-13 12:23:35,718 INFO [StoreOpener-e7ec1481802597d098b27d5a31dd86f7-1] regionserver.HStore(310): Store=e7ec1481802597d098b27d5a31dd86f7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:35,720 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,721 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,725 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:23:35,728 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:23:35,729 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e7ec1481802597d098b27d5a31dd86f7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72833450, jitterRate=0.08530297875404358}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:23:35,729 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e7ec1481802597d098b27d5a31dd86f7: Running coprocessor pre-open hook at 1676291015713Writing region info on filesystem at 1676291015713Initializing all the Stores at 1676291015714 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676291015714Cleaning up temporary data from old regions at 1676291015722 (+8 ms)Cleaning up detritus from prior splits at 1676291015722Running coprocessor post-open hooks at 1676291015729 (+7 ms)Region opened successfully at 1676291015729 2023-02-13 12:23:35,731 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7., pid=6, masterSystemTime=1676291015704 2023-02-13 12:23:35,735 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,736 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:23:35,737 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e7ec1481802597d098b27d5a31dd86f7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:35,738 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676291015736"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291015736"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291015736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291015736"}]},"ts":"1676291015736"} 2023-02-13 12:23:35,746 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 12:23:35,746 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e7ec1481802597d098b27d5a31dd86f7, server=jenkins-hbase16.apache.org,38471,1676291011293 in 192 msec 2023-02-13 12:23:35,750 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 12:23:35,751 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e7ec1481802597d098b27d5a31dd86f7, ASSIGN in 361 msec 2023-02-13 12:23:35,752 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:23:35,752 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291015752"}]},"ts":"1676291015752"} 2023-02-13 12:23:35,755 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 12:23:35,776 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:23:35,781 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 977 msec 2023-02-13 12:23:35,811 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 12:23:35,837 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:23:35,837 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:23:35,885 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 12:23:35,911 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:23:35,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 49 msec 2023-02-13 12:23:35,928 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 12:23:35,945 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:23:35,957 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 28 msec 2023-02-13 12:23:35,978 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 12:23:35,995 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 12:23:35,995 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 4.603sec 2023-02-13 12:23:35,997 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 12:23:35,999 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 12:23:35,999 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 12:23:36,000 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41493,1676291010224-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 12:23:36,002 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,41493,1676291010224-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 12:23:36,013 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 12:23:36,042 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(139): Connect 0x6f2ea739 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:36,054 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@410bdb3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:36,071 DEBUG [hconnection-0x716f2b59-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:23:36,091 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43776, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:23:36,118 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:23:36,119 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(139): Connect 0x649a9174 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:36,142 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76d416b1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:36,142 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:23:36,168 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=211, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=171, AvailableMemoryMB=4792 2023-02-13 12:23:36,178 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:23:36,182 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:55842, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:23:36,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:23:36,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:36,196 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:23:36,198 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:23:36,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-13 12:23:36,201 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:36,203 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 empty. 2023-02-13 12:23:36,205 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:36,205 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-13 12:23:36,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 12:23:36,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 12:23:36,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 12:23:36,627 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:36,631 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => da1f57ab313c133ba31c2c29865561b6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:23:36,643 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:36,643 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing da1f57ab313c133ba31c2c29865561b6, disabling compactions & flushes 2023-02-13 12:23:36,643 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:36,643 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:36,643 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. after waiting 0 ms 2023-02-13 12:23:36,643 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:36,644 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:36,644 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for da1f57ab313c133ba31c2c29865561b6: Waiting for close lock at 1676291016643Disabling compacts and flushes for region at 1676291016643Disabling writes for close at 1676291016643Writing region close event to WAL at 1676291016644 (+1 ms)Closed at 1676291016644 2023-02-13 12:23:36,648 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:23:36,650 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291016650"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291016650"}]},"ts":"1676291016650"} 2023-02-13 12:23:36,653 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:23:36,655 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:23:36,655 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291016655"}]},"ts":"1676291016655"} 2023-02-13 12:23:36,658 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-13 12:23:36,700 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, ASSIGN}] 2023-02-13 12:23:36,703 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, ASSIGN 2023-02-13 12:23:36,705 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:23:36,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 12:23:36,857 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:36,857 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291016857"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291016857"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291016857"}]},"ts":"1676291016857"} 2023-02-13 12:23:36,861 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:37,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:37,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => da1f57ab313c133ba31c2c29865561b6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,028 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,031 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:37,031 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:37,034 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da1f57ab313c133ba31c2c29865561b6 columnFamilyName family1 2023-02-13 12:23:37,037 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(310): Store=da1f57ab313c133ba31c2c29865561b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:37,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:37,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:23:37,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened da1f57ab313c133ba31c2c29865561b6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61579351, jitterRate=-0.08239616453647614}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:23:37,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for da1f57ab313c133ba31c2c29865561b6: Running coprocessor pre-open hook at 1676291017023Writing region info on filesystem at 1676291017023Initializing all the Stores at 1676291017025 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291017025Cleaning up temporary data from old regions at 1676291017042 (+17 ms)Cleaning up detritus from prior splits at 1676291017043 (+1 ms)Running coprocessor post-open hooks at 1676291017052 (+9 ms)Region opened successfully at 1676291017052 2023-02-13 12:23:37,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., pid=11, masterSystemTime=1676291017015 2023-02-13 12:23:37,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:37,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:37,057 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:37,059 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291017057"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291017057"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291017057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291017057"}]},"ts":"1676291017057"} 2023-02-13 12:23:37,066 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 12:23:37,066 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293 in 202 msec 2023-02-13 12:23:37,069 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 12:23:37,071 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, ASSIGN in 366 msec 2023-02-13 12:23:37,072 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:23:37,072 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291017072"}]},"ts":"1676291017072"} 2023-02-13 12:23:37,074 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-13 12:23:37,135 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:23:37,139 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 943 msec 2023-02-13 12:23:37,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-13 12:23:37,328 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-13 12:23:37,349 DEBUG [hconnection-0x7b527796-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:23:37,353 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43784, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:23:37,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:37,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,504 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:37,504 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:37,505 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:37,505 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:37,506 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,506 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:37,515 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:37,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:37,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:37,516 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:37,516 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:37,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:37,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,518 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:37,520 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:37,520 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:37,521 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:37,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:37,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:37,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:37,523 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:37,524 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:23:38,065 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a3335254dae34dd5a2d496e830bccc68 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a3335254dae34dd5a2d496e830bccc68 2023-02-13 12:23:38,480 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/eec6b8996cc447fa8497bb189ebee151 2023-02-13 12:23:38,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/eec6b8996cc447fa8497bb189ebee151 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151 2023-02-13 12:23:38,507 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-13 12:23:38,510 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 985ms, sequenceid=5, compaction requested=false 2023-02-13 12:23:38,511 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:38,513 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:38,513 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:38,513 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:38,514 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:38,514 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,514 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:38,514 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:38,549 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,549 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:38,550 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,550 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:38,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:38,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:38,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:38,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:38,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7fa209c9[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:38,555 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:38,556 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,566 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,566 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:38,567 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:38,567 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:38,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:38,580 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:38,580 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,580 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:38,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:38,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:38,581 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:38,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:38,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:38,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:38,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:38,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:38,592 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:38,592 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:38,592 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:38,592 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,593 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:38,603 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,603 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:38,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,603 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:38,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:38,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:38,604 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:38,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:38,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,606 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:38,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,620 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,620 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:38,620 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:38,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:38,620 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:38,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:38,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:38,620 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:38,620 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,621 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,621 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,621 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,621 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:38,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:38,622 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:38,622 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:38,722 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:38,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:38,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:38,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:38,747 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,747 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:38,747 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:38,748 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:38,748 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:38,749 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,749 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,757 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:38,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:38,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:38,758 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:38,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,758 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:38,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:38,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:38,760 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:38,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:38,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:38,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:38,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:38,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:38,761 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:38,761 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:38,761 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:23:39,200 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ccbc34e78efe48799630faf9e043541e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ccbc34e78efe48799630faf9e043541e 2023-02-13 12:23:39,333 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 12:23:39,336 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 12:23:39,616 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/e2b283578a4a4cc5bcbb8886e5b13d18 2023-02-13 12:23:39,630 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/e2b283578a4a4cc5bcbb8886e5b13d18 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18 2023-02-13 12:23:39,638 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:23:39,640 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 879ms, sequenceid=9, compaction requested=false 2023-02-13 12:23:39,640 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:39,640 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:39,640 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:39,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:39,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:39,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:39,674 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,674 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:39,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:39,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:39,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:39,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:39,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:39,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:39,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d0f7e4a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:39,678 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:39,678 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,686 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,687 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:39,687 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:39,687 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:39,687 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,687 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:39,721 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:39,722 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:39,722 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:39,721 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:39,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:39,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:39,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:39,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:39,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:39,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:39,726 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:39,726 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:39,726 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:39,726 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,726 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:39,740 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,740 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,741 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:39,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:39,741 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:39,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,743 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:39,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:39,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:39,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:39,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:39,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:39,753 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,753 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,754 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,754 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,754 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,754 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:39,754 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:39,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:39,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:39,854 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:39,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:39,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:39,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:39,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,866 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:39,866 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:39,867 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:39,867 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:39,867 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,867 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,878 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:39,878 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:39,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:39,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:39,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:39,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:39,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:39,880 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:39,880 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:39,881 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:39,881 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:39,881 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:39,881 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:39,881 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:39,881 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:40,107 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:23:40,314 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120107faf63d691e47be8cf36906f5d36226 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120107faf63d691e47be8cf36906f5d36226 2023-02-13 12:23:40,352 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 12:23:40,353 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 12:23:40,731 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/cb7c251e929b492cae4ee40f8d0829a1 2023-02-13 12:23:40,747 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/cb7c251e929b492cae4ee40f8d0829a1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1 2023-02-13 12:23:40,755 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 12:23:40,756 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 875ms, sequenceid=13, compaction requested=true 2023-02-13 12:23:40,756 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:40,756 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:40,756 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:40,756 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:40,756 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,757 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:40,757 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:40,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,791 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:40,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:40,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:40,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:40,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:40,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:40,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:40,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16d7c050[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:40,795 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:40,796 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,803 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:40,803 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:40,804 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,804 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:40,816 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:40,816 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,816 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:40,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:40,816 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:40,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:40,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:40,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:40,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:40,822 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:40,822 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:40,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:40,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:40,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,822 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:40,832 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,832 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:40,832 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:40,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:40,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:40,833 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:40,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,834 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:40,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,844 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,844 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:40,845 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:40,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:40,845 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 12:23:40,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:40,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:40,845 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:40,845 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,845 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:40,845 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,846 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,846 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,846 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,846 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:40,846 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:40,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:40,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:40,946 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:40,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:40,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:40,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:40,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,963 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:40,963 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:40,964 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:40,964 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:40,965 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,965 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,974 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,974 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:40,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:40,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:40,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:40,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:40,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:40,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:40,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:40,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:40,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:40,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:40,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:40,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:40,978 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:40,978 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:40,978 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:41,416 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055ec733a301ef4d91b5dd5c9391ac7332 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055ec733a301ef4d91b5dd5c9391ac7332 2023-02-13 12:23:41,429 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/09c3bc6729334572a9aa7fafa8ce69af 2023-02-13 12:23:41,438 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/09c3bc6729334572a9aa7fafa8ce69af as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af 2023-02-13 12:23:41,445 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 12:23:41,447 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 469ms, sequenceid=17, compaction requested=true 2023-02-13 12:23:41,447 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:41,447 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:41,447 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:41,447 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:41,447 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,447 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:41,447 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:41,482 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,482 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:41,483 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,483 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:41,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:41,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:41,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:41,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:41,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e0eb8e7[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:41,485 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:41,485 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,494 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:41,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:41,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:41,495 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,495 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:41,503 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:41,503 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,503 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:41,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:41,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:41,503 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:41,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:41,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:41,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:41,507 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:41,507 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:41,507 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:41,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:41,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,508 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:41,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:41,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:41,516 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:41,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,518 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:41,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,532 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,532 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:41,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 12:23:41,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:41,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:41,532 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:41,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:41,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:41,532 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:41,532 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,533 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,533 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:41,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:41,533 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,534 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:41,534 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,534 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:41,534 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,634 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:41,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:41,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:41,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:41,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,654 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:41,654 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:41,655 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:41,655 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:41,655 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,655 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,665 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:41,666 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:41,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:41,666 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:41,666 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:41,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:41,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:41,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:41,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:41,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:41,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:41,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:41,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:41,670 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:41,670 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:41,670 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:42,104 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d4d83acad22c4cd39a58873d75cb3c42 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228d4d83acad22c4cd39a58873d75cb3c42 2023-02-13 12:23:42,117 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/e69149f10ffd4a8a89a7130d9bc8f89e 2023-02-13 12:23:42,126 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/e69149f10ffd4a8a89a7130d9bc8f89e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e 2023-02-13 12:23:42,134 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 12:23:42,136 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 466ms, sequenceid=21, compaction requested=true 2023-02-13 12:23:42,136 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:42,136 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:42,136 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:42,136 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:42,136 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,137 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:42,137 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:42,204 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,204 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,206 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,206 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:42,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:42,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@618d14ae[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:42,212 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:42,212 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,228 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,229 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:42,229 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:42,229 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:42,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:42,241 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:42,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:42,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:42,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:42,250 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:42,250 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:42,250 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:42,250 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,250 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:42,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,263 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:42,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,266 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:42,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-13 12:23:42,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:42,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:42,278 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:42,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:42,278 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:42,280 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:42,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,280 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:42,281 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:42,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:42,381 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:42,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:42,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:42,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:42,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,395 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:42,395 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:42,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:42,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:42,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:42,407 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:42,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:42,408 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,408 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:42,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:42,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:42,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:42,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:42,412 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:42,412 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:42,412 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:42,412 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:42,413 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:42,441 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313292a525ce55438d9589792cdb39be05 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512313292a525ce55438d9589792cdb39be05 2023-02-13 12:23:42,852 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/69a4138f55ae4ad58789079c4122036e 2023-02-13 12:23:42,868 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/69a4138f55ae4ad58789079c4122036e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e 2023-02-13 12:23:42,875 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 12:23:42,877 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 464ms, sequenceid=25, compaction requested=true 2023-02-13 12:23:42,877 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:42,877 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:42,877 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:42,878 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:42,878 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,878 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:42,878 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:42,915 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,915 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,916 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,916 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:42,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:42,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@796cbfb[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:42,918 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:42,918 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,928 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,928 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:42,928 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:42,928 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:42,928 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,928 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:42,936 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:42,937 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:42,943 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:42,943 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:42,943 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:42,943 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:42,943 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,943 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:42,953 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,953 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:42,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:42,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,953 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:42,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:42,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:42,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:42,955 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:42,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,958 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:42,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,969 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,969 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:42,969 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:42,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:42,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:42,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:42,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:42,969 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,970 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:42,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:42,971 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:42,970 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,071 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:43,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:43,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:43,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:43,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,089 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:43,089 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:43,090 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:43,090 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:43,090 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,090 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,099 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:43,099 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:43,099 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,100 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,102 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,102 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:43,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,103 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:43,103 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:43,103 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:43,104 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:43,104 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:43,104 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:43,104 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:43,104 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:43,538 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101da626edb0f674a83b4064716021c9f32 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101da626edb0f674a83b4064716021c9f32 2023-02-13 12:23:43,549 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/fdccd9327db3495e9ef7dd40273265a8 2023-02-13 12:23:43,561 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/fdccd9327db3495e9ef7dd40273265a8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8 2023-02-13 12:23:43,569 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 12:23:43,570 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 466ms, sequenceid=29, compaction requested=true 2023-02-13 12:23:43,570 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:43,570 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:43,570 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:43,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:43,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:43,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:43,607 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,607 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,608 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,608 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:43,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:43,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f6b4e19[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:43,614 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:43,614 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,624 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,624 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:43,625 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:43,625 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:43,625 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,625 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:43,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:43,636 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:43,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:43,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:43,643 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:43,643 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:43,643 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:43,643 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,643 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:43,653 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,653 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,653 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:43,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,654 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,657 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:43,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:43,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:43,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:43,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:43,689 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:43,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,691 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,691 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,691 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:43,691 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:43,691 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:43,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:43,692 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,792 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:43,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:43,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:43,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:43,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,812 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:43,812 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:43,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:43,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:43,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,828 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:43,828 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:43,829 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,829 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:43,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,830 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:43,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:43,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:43,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:43,832 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:43,832 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:43,832 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:43,832 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:43,832 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:43,862 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031697da76585741c0aa2187288ce2a6c2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601031697da76585741c0aa2187288ce2a6c2 2023-02-13 12:23:43,873 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/b9172b6783dd44769a0c2aed87adaa02 2023-02-13 12:23:43,880 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/b9172b6783dd44769a0c2aed87adaa02 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02 2023-02-13 12:23:43,888 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 12:23:43,889 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 57ms, sequenceid=33, compaction requested=true 2023-02-13 12:23:43,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:43,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:43,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:43,890 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:43,890 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,890 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:43,890 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:43,899 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,899 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,899 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,899 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:43,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54d91293[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:43,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:43,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,911 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:43,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,912 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:43,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:43,919 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:43,919 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:43,919 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,919 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:43,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:43,923 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:43,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:43,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:43,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,923 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:43,932 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,932 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:43,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:43,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,932 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:43,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:43,932 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,934 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:43,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,944 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,944 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:43,944 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:43,944 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:43,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:43,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:43,945 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:43,945 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:43,946 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:44,047 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:44,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:44,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:44,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:44,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,064 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:44,064 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:44,064 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:44,064 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:44,065 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,065 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,074 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:44,074 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:44,074 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,075 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,075 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,076 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:44,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,076 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:44,076 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:44,077 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:44,077 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:44,077 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:44,077 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:44,077 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:44,077 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:44,508 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075d65c56a0f8b4290aa04500f43c8b9b4 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509075d65c56a0f8b4290aa04500f43c8b9b4 2023-02-13 12:23:44,954 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/2099f8eee0e44609ae8af18bcf4a032e 2023-02-13 12:23:44,966 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/2099f8eee0e44609ae8af18bcf4a032e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e 2023-02-13 12:23:44,976 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 12:23:44,977 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 900ms, sequenceid=37, compaction requested=true 2023-02-13 12:23:44,977 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:44,977 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:44,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:44,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:44,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:44,978 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,986 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,986 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:44,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:44,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5df844c2[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:44,988 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:44,988 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,999 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:44,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:44,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:44,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:44,999 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:44,999 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:45,007 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:45,007 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:45,007 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:45,007 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:45,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:45,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:45,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:45,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:45,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:45,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:45,015 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:45,015 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:45,015 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:45,015 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,015 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:45,028 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,028 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:45,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,028 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:45,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:45,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:45,029 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,033 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,044 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,044 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:45,044 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,044 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:45,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:45,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:45,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:45,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:45,045 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:45,045 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:45,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,046 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,146 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:45,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:45,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:45,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:45,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,162 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:45,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:45,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-13 12:23:45,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-13 12:23:45,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,169 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,169 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:45,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:45,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:45,170 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:45,170 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:45,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:45,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-13 12:23:45,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:45,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-13 12:23:45,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:45,172 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-13 12:23:45,173 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:45,173 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:45,173 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:45,173 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. started... 2023-02-13 12:23:45,173 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing da1f57ab313c133ba31c2c29865561b6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:45,604 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120aae4c19a430e46c6941f4cb633ec6973 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120aae4c19a430e46c6941f4cb633ec6973 2023-02-13 12:23:46,013 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/4a279e7f5feb492c97bbecdb89b490a5 2023-02-13 12:23:46,026 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/4a279e7f5feb492c97bbecdb89b490a5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5 2023-02-13 12:23:46,033 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 12:23:46,035 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for da1f57ab313c133ba31c2c29865561b6 in 862ms, sequenceid=41, compaction requested=true 2023-02-13 12:23:46,035 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:46,035 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-13 12:23:46,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-13 12:23:46,074 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,074 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:46,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:46,075 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,075 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:46,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-13 12:23:46,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c008595[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:46,077 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-13 12:23:46,077 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,107 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,107 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-13 12:23:46,107 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-13 12:23:46,107 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,107 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:46,107 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:46,124 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-13 12:23:46,124 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,124 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:46,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:46,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:46,124 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-13 12:23:46,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:46,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:46,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-13 12:23:46,128 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-13 12:23:46,128 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:46,128 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:46,128 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,128 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:46,136 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,136 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:46,136 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:46,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:46,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:46,137 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,138 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-13 12:23:46,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:46,149 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,149 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:46,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:46,150 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:46,250 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:46,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:46,256 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:23:46,259 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44256, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:23:46,260 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:46,275 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:46,276 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:46,278 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:23:46,278 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291626278 2023-02-13 12:23:46,279 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:23:46,284 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 12:23:46,284 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:46,285 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x1335b40e to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:46,299 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a347cdb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:46,301 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,384 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010363c7bfac7f114df5be9d775e556fd719 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010363c7bfac7f114df5be9d775e556fd719 2023-02-13 12:23:46,386 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:46,404 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:23:46,406 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:38472, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:23:46,417 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:23:46,419 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44258, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:23:46,458 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a361f67f44f94c18ba138d177a655748 first=Optional[r3] last=Optional[r6] 2023-02-13 12:23:46,470 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a361f67f44f94c18ba138d177a655748}] 2023-02-13 12:23:46,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a361f67f44f94c18ba138d177a655748 for inclusion in da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 12:23:46,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:23:46,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:46,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/a361f67f44f94c18ba138d177a655748 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kbc30o0jdl5j54r8ht6i0gcrups4tohbn2m0p9u87vnc9b8kngr08jvu52q0p9e2/family1/a361f67f44f94c18ba138d177a655748 2023-02-13 12:23:46,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kbc30o0jdl5j54r8ht6i0gcrups4tohbn2m0p9u87vnc9b8kngr08jvu52q0p9e2/family1/a361f67f44f94c18ba138d177a655748 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_ 2023-02-13 12:23:46,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kbc30o0jdl5j54r8ht6i0gcrups4tohbn2m0p9u87vnc9b8kngr08jvu52q0p9e2/family1/a361f67f44f94c18ba138d177a655748 into da1f57ab313c133ba31c2c29865561b6/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_ - updating store file list. 2023-02-13 12:23:46,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_ into da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:46,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kbc30o0jdl5j54r8ht6i0gcrups4tohbn2m0p9u87vnc9b8kngr08jvu52q0p9e2/family1/a361f67f44f94c18ba138d177a655748 into da1f57ab313c133ba31c2c29865561b6/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_) 2023-02-13 12:23:46,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__kbc30o0jdl5j54r8ht6i0gcrups4tohbn2m0p9u87vnc9b8kngr08jvu52q0p9e2/family1/a361f67f44f94c18ba138d177a655748 2023-02-13 12:23:46,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:23:46,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:23:46,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): da1f57ab313c133ba31c2c29865561b6/family1 is initiating minor compaction 2023-02-13 12:23:46,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3cab8268; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:23:46,528 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of da1f57ab313c133ba31c2c29865561b6/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:46,529 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp, totalSize=50.2 K 2023-02-13 12:23:46,529 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e2b283578a4a4cc5bcbb8886e5b13d18, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting cb7c251e929b492cae4ee40f8d0829a1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 09c3bc6729334572a9aa7fafa8ce69af, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e69149f10ffd4a8a89a7130d9bc8f89e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 69a4138f55ae4ad58789079c4122036e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting fdccd9327db3495e9ef7dd40273265a8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting b9172b6783dd44769a0c2aed87adaa02, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 2099f8eee0e44609ae8af18bcf4a032e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:23:46,530 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 4a279e7f5feb492c97bbecdb89b490a5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:23:46,531 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e57704277e474b60a39adeca5171c23f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:23:46,544 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:46,546 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:23:46,551 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228d4d83acad22c4cd39a58873d75cb3c42 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228d4d83acad22c4cd39a58873d75cb3c42 2023-02-13 12:23:46,553 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512313292a525ce55438d9589792cdb39be05 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512313292a525ce55438d9589792cdb39be05 2023-02-13 12:23:46,555 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101da626edb0f674a83b4064716021c9f32 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101da626edb0f674a83b4064716021c9f32 2023-02-13 12:23:46,557 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601031697da76585741c0aa2187288ce2a6c2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601031697da76585741c0aa2187288ce2a6c2 2023-02-13 12:23:46,558 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-13 12:23:46,558 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,559 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:46,592 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): da1f57ab313c133ba31c2c29865561b6#family1#compaction#10 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:23:47,060 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051b999b95622e4ce8ad5a90d9e35873de to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512051b999b95622e4ce8ad5a90d9e35873de 2023-02-13 12:23:47,062 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:47,095 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0edd2625b21a4701b13260f31ff12e35 first=Optional[r00] last=Optional[r2] 2023-02-13 12:23:47,096 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0edd2625b21a4701b13260f31ff12e35}] 2023-02-13 12:23:47,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0edd2625b21a4701b13260f31ff12e35 for inclusion in da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:47,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 12:23:47,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:23:47,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:47,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/0edd2625b21a4701b13260f31ff12e35 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__othdp8g1uveuaqq4t7sasade4vero4pglfe7n2c6foh9k5ejln7vvd55iluo14rr/family1/0edd2625b21a4701b13260f31ff12e35 2023-02-13 12:23:47,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__othdp8g1uveuaqq4t7sasade4vero4pglfe7n2c6foh9k5ejln7vvd55iluo14rr/family1/0edd2625b21a4701b13260f31ff12e35 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ 2023-02-13 12:23:47,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__othdp8g1uveuaqq4t7sasade4vero4pglfe7n2c6foh9k5ejln7vvd55iluo14rr/family1/0edd2625b21a4701b13260f31ff12e35 into da1f57ab313c133ba31c2c29865561b6/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ - updating store file list. 2023-02-13 12:23:47,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ into da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:47,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__othdp8g1uveuaqq4t7sasade4vero4pglfe7n2c6foh9k5ejln7vvd55iluo14rr/family1/0edd2625b21a4701b13260f31ff12e35 into da1f57ab313c133ba31c2c29865561b6/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_) 2023-02-13 12:23:47,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__othdp8g1uveuaqq4t7sasade4vero4pglfe7n2c6foh9k5ejln7vvd55iluo14rr/family1/0edd2625b21a4701b13260f31ff12e35 2023-02-13 12:23:47,117 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:47,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:23:47,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a3335254dae34dd5a2d496e830bccc68 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a3335254dae34dd5a2d496e830bccc68 2023-02-13 12:23:47,123 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120107faf63d691e47be8cf36906f5d36226 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120107faf63d691e47be8cf36906f5d36226 2023-02-13 12:23:47,125 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055ec733a301ef4d91b5dd5c9391ac7332 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512055ec733a301ef4d91b5dd5c9391ac7332 2023-02-13 12:23:47,125 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-13 12:23:47,125 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 12:23:47,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:47,125 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:47,126 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:23:47,127 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:47,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 861 msec 2023-02-13 12:23:47,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 12:23:47,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:47,426 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/ca13b748fdc04096ba56c60da8fed831 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831 2023-02-13 12:23:47,438 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in da1f57ab313c133ba31c2c29865561b6/family1 of da1f57ab313c133ba31c2c29865561b6 into ca13b748fdc04096ba56c60da8fed831(size=6.6 K), total size for store is 358.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:23:47,438 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:47,440 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1676291026526; duration=0sec 2023-02-13 12:23:47,440 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:23:47,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_] to archive 2023-02-13 12:23:47,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:23:47,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e2b283578a4a4cc5bcbb8886e5b13d18 2023-02-13 12:23:47,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/cb7c251e929b492cae4ee40f8d0829a1 2023-02-13 12:23:47,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/09c3bc6729334572a9aa7fafa8ce69af 2023-02-13 12:23:47,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e69149f10ffd4a8a89a7130d9bc8f89e 2023-02-13 12:23:47,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/69a4138f55ae4ad58789079c4122036e 2023-02-13 12:23:47,487 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/fdccd9327db3495e9ef7dd40273265a8 2023-02-13 12:23:47,490 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/b9172b6783dd44769a0c2aed87adaa02 2023-02-13 12:23:47,491 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2099f8eee0e44609ae8af18bcf4a032e 2023-02-13 12:23:47,493 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/4a279e7f5feb492c97bbecdb89b490a5 2023-02-13 12:23:47,494 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/e57704277e474b60a39adeca5171c23f_SeqId_44_ 2023-02-13 12:23:47,713 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 12:23:47,715 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:47,716 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-13 12:23:47,722 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-13 12:23:47,736 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, REOPEN/MOVE}] 2023-02-13 12:23:47,737 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, REOPEN/MOVE 2023-02-13 12:23:47,739 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:47,739 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291027739"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291027739"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291027739"}]},"ts":"1676291027739"} 2023-02-13 12:23:47,742 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:47,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:47,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing da1f57ab313c133ba31c2c29865561b6, disabling compactions & flushes 2023-02-13 12:23:47,907 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:47,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:47,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. after waiting 0 ms 2023-02-13 12:23:47,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:47,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-13 12:23:47,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:47,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for da1f57ab313c133ba31c2c29865561b6: Waiting for close lock at 1676291027906Running coprocessor pre-close hooks at 1676291027906Disabling compacts and flushes for region at 1676291027906Disabling writes for close at 1676291027907 (+1 ms)Writing region close event to WAL at 1676291027914 (+7 ms)Running coprocessor post-close hooks at 1676291027920 (+6 ms)Closed at 1676291027921 (+1 ms) 2023-02-13 12:23:47,921 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: da1f57ab313c133ba31c2c29865561b6 to self. 2023-02-13 12:23:47,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:47,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=CLOSED 2023-02-13 12:23:47,925 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291027924"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291027924"}]},"ts":"1676291027924"} 2023-02-13 12:23:47,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-13 12:23:47,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293 in 185 msec 2023-02-13 12:23:47,930 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=true 2023-02-13 12:23:48,081 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:48,082 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291028081"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291028081"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291028081"}]},"ts":"1676291028081"} 2023-02-13 12:23:48,088 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:48,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:48,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => da1f57ab313c133ba31c2c29865561b6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:48,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:48,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,249 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,250 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:48,251 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:48,251 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region da1f57ab313c133ba31c2c29865561b6 columnFamilyName family1 2023-02-13 12:23:48,262 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ 2023-02-13 12:23:48,269 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831 2023-02-13 12:23:48,274 DEBUG [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151 2023-02-13 12:23:48,274 INFO [StoreOpener-da1f57ab313c133ba31c2c29865561b6-1] regionserver.HStore(310): Store=da1f57ab313c133ba31c2c29865561b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:48,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:23:48,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened da1f57ab313c133ba31c2c29865561b6; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59139260, jitterRate=-0.11875635385513306}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:23:48,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for da1f57ab313c133ba31c2c29865561b6: Running coprocessor pre-open hook at 1676291028248Writing region info on filesystem at 1676291028248Initializing all the Stores at 1676291028249 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676291028249Cleaning up temporary data from old regions at 1676291028278 (+29 ms)Cleaning up detritus from prior splits at 1676291028278Running coprocessor post-open hooks at 1676291028281 (+3 ms)Region opened successfully at 1676291028281 2023-02-13 12:23:48,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., pid=17, masterSystemTime=1676291028243 2023-02-13 12:23:48,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:23:48,284 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:23:48,286 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:23:48,286 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1912): da1f57ab313c133ba31c2c29865561b6/family1 is initiating minor compaction (all files) 2023-02-13 12:23:48,286 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of da1f57ab313c133ba31c2c29865561b6/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:48,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:48,286 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp, totalSize=358.6 K 2023-02-13 12:23:48,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:23:48,287 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=da1f57ab313c133ba31c2c29865561b6, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:48,287 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting eec6b8996cc447fa8497bb189ebee151, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:23:48,288 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676291028287"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291028287"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291028287"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291028287"}]},"ts":"1676291028287"} 2023-02-13 12:23:48,288 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting ca13b748fdc04096ba56c60da8fed831, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:23:48,288 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 33861a649ad340f089a8c02de9de51b5_SeqId_46_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878836144 2023-02-13 12:23:48,293 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 12:23:48,293 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure da1f57ab313c133ba31c2c29865561b6, server=jenkins-hbase16.apache.org,38471,1676291011293 in 202 msec 2023-02-13 12:23:48,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-13 12:23:48,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=da1f57ab313c133ba31c2c29865561b6, REOPEN/MOVE in 557 msec 2023-02-13 12:23:48,300 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 12:23:48,300 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 577 msec 2023-02-13 12:23:48,304 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0150 sec 2023-02-13 12:23:48,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-13 12:23:48,318 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(278): alter status finished 2023-02-13 12:23:48,319 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-13 12:23:48,324 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:48,325 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:48,326 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:23:48,326 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291628326 2023-02-13 12:23:48,327 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:23:48,328 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-13 12:23:48,328 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:48,328 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x5af77c43 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:23:48,328 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): da1f57ab313c133ba31c2c29865561b6#family1#compaction#11 average throughput is 5.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:23:48,375 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d92f587, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:23:48,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:48,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:48,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:48,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:48,375 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:49,214 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f79e8b6d7cc04d1a8cd100f9dc3aca0e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128f79e8b6d7cc04d1a8cd100f9dc3aca0e 2023-02-13 12:23:49,216 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:49,231 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:23:49,232 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:38480, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:23:49,240 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:23:49,242 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44262, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:23:49,255 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9276425784be438cadacdc1297feeda1 first=Optional[r05] last=Optional[r8] 2023-02-13 12:23:49,256 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=50 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9276425784be438cadacdc1297feeda1}] 2023-02-13 12:23:49,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9276425784be438cadacdc1297feeda1 for inclusion in da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:49,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 12:23:49,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:23:49,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:49,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9276425784be438cadacdc1297feeda1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ieoh2i6lh0d2t6n46tu0ksm24sgjsmpbtdb96r4ftv3de3643ro3lj0o5nijae5c/family1/9276425784be438cadacdc1297feeda1 2023-02-13 12:23:49,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ieoh2i6lh0d2t6n46tu0ksm24sgjsmpbtdb96r4ftv3de3643ro3lj0o5nijae5c/family1/9276425784be438cadacdc1297feeda1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/677c1159f5b149a69eb0308584f1e47f_SeqId_52_ 2023-02-13 12:23:49,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ieoh2i6lh0d2t6n46tu0ksm24sgjsmpbtdb96r4ftv3de3643ro3lj0o5nijae5c/family1/9276425784be438cadacdc1297feeda1 into da1f57ab313c133ba31c2c29865561b6/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/677c1159f5b149a69eb0308584f1e47f_SeqId_52_ - updating store file list. 2023-02-13 12:23:49,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/677c1159f5b149a69eb0308584f1e47f_SeqId_52_ into da1f57ab313c133ba31c2c29865561b6/family1 2023-02-13 12:23:49,278 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ieoh2i6lh0d2t6n46tu0ksm24sgjsmpbtdb96r4ftv3de3643ro3lj0o5nijae5c/family1/9276425784be438cadacdc1297feeda1 into da1f57ab313c133ba31c2c29865561b6/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/677c1159f5b149a69eb0308584f1e47f_SeqId_52_) 2023-02-13 12:23:49,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ieoh2i6lh0d2t6n46tu0ksm24sgjsmpbtdb96r4ftv3de3643ro3lj0o5nijae5c/family1/9276425784be438cadacdc1297feeda1 2023-02-13 12:23:49,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:49,283 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:23:49,285 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120aae4c19a430e46c6941f4cb633ec6973 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120aae4c19a430e46c6941f4cb633ec6973 2023-02-13 12:23:49,287 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ccbc34e78efe48799630faf9e043541e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ccbc34e78efe48799630faf9e043541e 2023-02-13 12:23:49,287 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-13 12:23:49,287 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 12:23:49,287 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:49,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:49,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-13 12:23:49,288 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-13 12:23:49,288 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:23:49,289 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-13 12:23:49,292 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 969 msec 2023-02-13 12:23:49,394 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=250 (was 211) Potentially hanging thread: hconnection-0x282b01e8-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x282b01e8-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52126 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52028 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x282b01e8-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1335b40e-SendThread(127.0.0.1:64508) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52056 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase16:38471-shortCompactions-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:940) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:880) org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:863) org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:818) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishClose(HFileWriterImpl.java:820) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:654) org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:359) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:89) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.commitWriter(DefaultCompactor.java:42) org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:362) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:108) org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122) org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1473) org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2266) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51948 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52006 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x282b01e8-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-616355569_15 at /127.0.0.1:51918 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51974 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x282b01e8-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51978 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51992 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1335b40e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x5af77c43-SendThread(127.0.0.1:64508) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51958 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x5af77c43 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/349895967.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e4fdc64-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52138 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51982 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e4fdc64-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e4fdc64-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:52020 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x5af77c43-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/2c98876ff4e5427eb9d51d0153667dd0 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:51968 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1335b40e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/349895967.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=592 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=234 (was 259), ProcessCount=171 (was 171), AvailableMemoryMB=4670 (was 4792) 2023-02-13 12:23:49,411 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=250, OpenFileDescriptor=592, MaxFileDescriptor=60000, SystemLoadAverage=234, ProcessCount=171, AvailableMemoryMB=4667 2023-02-13 12:23:49,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 12:23:49,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:49,417 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:23:49,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-13 12:23:49,419 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:23:49,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:49,421 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:49,421 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765 empty. 2023-02-13 12:23:49,422 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:49,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-13 12:23:49,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:49,561 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/.tmp/family1/2c98876ff4e5427eb9d51d0153667dd0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/2c98876ff4e5427eb9d51d0153667dd0 2023-02-13 12:23:49,569 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in da1f57ab313c133ba31c2c29865561b6/family1 of da1f57ab313c133ba31c2c29865561b6 into 2c98876ff4e5427eb9d51d0153667dd0(size=177.2 K), total size for store is 182.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:23:49,569 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for da1f57ab313c133ba31c2c29865561b6: 2023-02-13 12:23:49,569 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6., storeName=da1f57ab313c133ba31c2c29865561b6/family1, priority=13, startTime=1676291028283; duration=1sec 2023-02-13 12:23:49,569 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:23:49,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_] to archive 2023-02-13 12:23:49,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:23:49,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/eec6b8996cc447fa8497bb189ebee151 2023-02-13 12:23:49,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/ca13b748fdc04096ba56c60da8fed831 2023-02-13 12:23:49,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/family1/33861a649ad340f089a8c02de9de51b5_SeqId_46_ 2023-02-13 12:23:49,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:49,834 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 12:23:49,837 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5a064ab3d6bc91c5da44588fa6275765, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:23:50,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:50,249 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:50,249 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 5a064ab3d6bc91c5da44588fa6275765, disabling compactions & flushes 2023-02-13 12:23:50,249 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,249 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,249 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. after waiting 0 ms 2023-02-13 12:23:50,250 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,250 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,250 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 5a064ab3d6bc91c5da44588fa6275765: Waiting for close lock at 1676291030249Disabling compacts and flushes for region at 1676291030249Disabling writes for close at 1676291030250 (+1 ms)Writing region close event to WAL at 1676291030250Closed at 1676291030250 2023-02-13 12:23:50,252 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:23:50,253 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291030253"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291030253"}]},"ts":"1676291030253"} 2023-02-13 12:23:50,255 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:23:50,256 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:23:50,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291030256"}]},"ts":"1676291030256"} 2023-02-13 12:23:50,258 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 12:23:50,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a064ab3d6bc91c5da44588fa6275765, ASSIGN}] 2023-02-13 12:23:50,302 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a064ab3d6bc91c5da44588fa6275765, ASSIGN 2023-02-13 12:23:50,304 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a064ab3d6bc91c5da44588fa6275765, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:23:50,457 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5a064ab3d6bc91c5da44588fa6275765, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:50,457 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291030456"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291030456"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291030456"}]},"ts":"1676291030456"} 2023-02-13 12:23:50,463 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 5a064ab3d6bc91c5da44588fa6275765, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:23:50,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:50,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5a064ab3d6bc91c5da44588fa6275765, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:23:50,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:23:50,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,631 INFO [StoreOpener-5a064ab3d6bc91c5da44588fa6275765-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,634 DEBUG [StoreOpener-5a064ab3d6bc91c5da44588fa6275765-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:23:50,634 DEBUG [StoreOpener-5a064ab3d6bc91c5da44588fa6275765-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:23:50,635 INFO [StoreOpener-5a064ab3d6bc91c5da44588fa6275765-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5a064ab3d6bc91c5da44588fa6275765 columnFamilyName family1 2023-02-13 12:23:50,638 INFO [StoreOpener-5a064ab3d6bc91c5da44588fa6275765-1] regionserver.HStore(310): Store=5a064ab3d6bc91c5da44588fa6275765/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:23:50,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:23:50,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:23:50,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5a064ab3d6bc91c5da44588fa6275765; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60030278, jitterRate=-0.1054791510105133}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:23:50,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5a064ab3d6bc91c5da44588fa6275765: Running coprocessor pre-open hook at 1676291030628Writing region info on filesystem at 1676291030628Initializing all the Stores at 1676291030630 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676291030630Cleaning up temporary data from old regions at 1676291030643 (+13 ms)Cleaning up detritus from prior splits at 1676291030643Running coprocessor post-open hooks at 1676291030650 (+7 ms)Region opened successfully at 1676291030650 2023-02-13 12:23:50,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., pid=21, masterSystemTime=1676291030617 2023-02-13 12:23:50,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:50,653 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5a064ab3d6bc91c5da44588fa6275765, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:50,653 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291030653"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291030653"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291030653"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291030653"}]},"ts":"1676291030653"} 2023-02-13 12:23:50,657 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-13 12:23:50,657 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 5a064ab3d6bc91c5da44588fa6275765, server=jenkins-hbase16.apache.org,38471,1676291011293 in 192 msec 2023-02-13 12:23:50,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 12:23:50,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=5a064ab3d6bc91c5da44588fa6275765, ASSIGN in 358 msec 2023-02-13 12:23:50,659 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:23:50,659 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291030659"}]},"ts":"1676291030659"} 2023-02-13 12:23:50,661 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 12:23:50,700 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:23:50,704 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.2870 sec 2023-02-13 12:23:51,341 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:51,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 12:23:51,534 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-13 12:23:51,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:51,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,573 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:51,574 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:51,574 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:51,574 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:51,575 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,575 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,617 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:51,617 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:51,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:51,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:51,618 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:51,618 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:51,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:51,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,620 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:51,621 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:51,621 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:51,622 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:51,622 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:51,622 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:51,622 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:51,622 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:51,623 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:23:52,068 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f93c8287cf204dbc97211be873ada177 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130f93c8287cf204dbc97211be873ada177 2023-02-13 12:23:52,476 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/49069fbec3504a41b39800890292c812 2023-02-13 12:23:52,486 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/49069fbec3504a41b39800890292c812 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812 2023-02-13 12:23:52,493 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 12:23:52,494 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 871ms, sequenceid=5, compaction requested=false 2023-02-13 12:23:52,495 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:52,495 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:52,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:52,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:52,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:52,495 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:52,532 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,532 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:52,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:52,533 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,533 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:52,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:52,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:52,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:52,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:52,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19ee058a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:52,537 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:52,538 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,548 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,549 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:52,549 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:52,549 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:52,549 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,549 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:52,557 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,557 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:52,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:52,557 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:52,557 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:52,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:52,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:52,560 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:52,560 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:52,560 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:52,560 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,560 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:52,569 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,569 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:52,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:52,569 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:52,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:52,570 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:52,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:52,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:52,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,571 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:52,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,582 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,582 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:52,582 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:52,582 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:52,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:52,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:23:52,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:52,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:52,582 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:52,582 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,583 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,583 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:52,583 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,583 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:52,584 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:52,584 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:52,684 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:52,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:52,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:52,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:52,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,702 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:52,702 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:52,702 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:52,703 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:52,703 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,703 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,711 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:52,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:52,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:52,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:52,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:52,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:52,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:52,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:52,714 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:52,714 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:52,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:52,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:52,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:52,715 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:52,715 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:52,715 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:23:53,144 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a010dfc8ab1843c1ae0abfc67ef87551 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a010dfc8ab1843c1ae0abfc67ef87551 2023-02-13 12:23:53,450 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:23:53,553 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/1959d955c0e344818dec2ed74b98bebe 2023-02-13 12:23:53,562 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/1959d955c0e344818dec2ed74b98bebe as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe 2023-02-13 12:23:53,570 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:23:53,571 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 856ms, sequenceid=9, compaction requested=false 2023-02-13 12:23:53,571 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:53,571 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:53,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:53,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:53,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,572 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:53,572 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:53,615 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,615 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:53,616 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,617 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:53,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:53,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bfc1ae[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:53,621 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:53,622 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,632 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,632 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:53,632 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:53,632 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:53,632 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,633 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:53,640 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,640 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:53,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:53,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:53,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:53,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:53,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:53,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:53,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:53,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:53,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:53,650 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:53,651 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:53,651 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:53,651 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,651 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:53,661 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,661 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:53,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:53,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,661 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:53,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:53,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:53,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:53,662 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:53,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,666 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:53,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,693 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,693 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:53,693 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:53,693 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:23:53,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:53,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:53,693 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:53,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:53,693 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:53,695 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,695 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:53,695 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,695 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:53,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:53,695 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,696 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:53,796 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:53,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:53,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:53,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:53,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,818 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:53,818 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:53,819 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:53,819 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:53,820 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,820 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,832 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,832 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:53,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:53,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:53,833 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:53,833 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:53,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:53,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:53,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:53,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:53,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:53,837 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:53,837 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:53,837 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:53,837 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:53,837 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:53,838 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:54,268 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f9578b15ac4e4d77af8e06f69865733e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f9578b15ac4e4d77af8e06f69865733e 2023-02-13 12:23:54,676 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/2ccb64edeace4d9988993db0e3e34a0c 2023-02-13 12:23:54,692 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/2ccb64edeace4d9988993db0e3e34a0c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c 2023-02-13 12:23:54,700 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 12:23:54,701 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 863ms, sequenceid=13, compaction requested=true 2023-02-13 12:23:54,701 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:54,701 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:54,701 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:54,701 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:54,701 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,701 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:54,701 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:54,732 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,732 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:54,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:54,732 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,732 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:54,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:54,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:54,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:54,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a7e4b8f[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:54,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:54,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,744 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,744 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:54,744 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:54,744 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:54,745 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,745 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:54,752 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:54,752 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:54,753 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:54,752 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:54,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:54,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:54,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:54,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:54,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:54,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:54,760 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:54,761 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:54,761 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:54,761 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,761 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:54,769 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,769 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:54,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:54,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,769 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:54,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:54,770 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:54,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,773 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:54,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:54,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:54,786 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:54,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:54,787 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:54,787 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:54,887 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:54,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:54,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:54,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:54,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,901 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:54,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:54,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:54,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:54,902 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,902 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,910 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:54,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:54,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:54,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:54,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:54,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:54,913 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:54,913 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:54,913 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:55,338 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d9069a32a75c42d18dda8119dcb49f20 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205d9069a32a75c42d18dda8119dcb49f20 2023-02-13 12:23:55,748 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/e351bbfa6d974fb69ca679871020560f 2023-02-13 12:23:55,764 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/e351bbfa6d974fb69ca679871020560f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f 2023-02-13 12:23:55,772 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 12:23:55,773 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 860ms, sequenceid=17, compaction requested=true 2023-02-13 12:23:55,773 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:55,773 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:55,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:55,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:55,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:55,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:55,807 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,807 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:55,807 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:55,807 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:55,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:55,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72ba4978[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:55,810 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:55,810 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,819 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,820 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:55,820 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,820 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:55,820 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:55,820 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:55,827 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:55,827 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:55,827 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,827 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:55,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:55,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:55,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:55,830 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:55,830 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:55,830 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:55,830 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,830 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:55,840 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,840 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:55,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:55,840 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:55,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,841 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:55,841 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:55,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:55,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:55,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:55,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:55,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:55,852 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:55,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:23:55,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,853 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:55,853 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,853 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:55,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:55,854 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:55,854 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,854 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,954 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:55,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:55,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:55,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:55,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,969 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:55,970 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:55,970 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:55,970 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:55,971 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,971 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,977 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,977 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:55,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:55,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:55,978 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:55,978 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:55,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:55,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:55,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:55,979 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:55,979 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:55,980 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:55,980 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:55,980 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:55,980 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:55,980 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:55,981 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:56,409 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288d7bbd62a50d42bc9703938140f7c906 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512288d7bbd62a50d42bc9703938140f7c906 2023-02-13 12:23:56,819 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/e1a7ff4f4b5746c1af93a235d8f6bc19 2023-02-13 12:23:56,833 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/e1a7ff4f4b5746c1af93a235d8f6bc19 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19 2023-02-13 12:23:56,840 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 12:23:56,841 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 860ms, sequenceid=21, compaction requested=true 2023-02-13 12:23:56,841 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:56,841 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:56,841 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:56,841 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:56,841 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,841 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:56,841 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:56,873 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,873 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:56,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:56,874 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,874 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:56,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:56,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:56,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:56,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:56,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b11c0f0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:56,878 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:56,878 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,886 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,886 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:56,886 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:56,886 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:56,886 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,887 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:56,894 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:56,894 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:56,894 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:56,894 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:56,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:56,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:56,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:56,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:56,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:56,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:56,900 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:56,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:56,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:56,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,901 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:56,928 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,928 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:56,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:56,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,928 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:56,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:56,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:56,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:56,929 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,932 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:56,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,949 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,949 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:56,949 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,949 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:56,949 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:56,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:56,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:56,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:56,949 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:56,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:56,951 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,951 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:56,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:56,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:56,951 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:56,952 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:56,952 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,052 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:57,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:57,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:57,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:57,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,071 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:57,071 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:57,071 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:57,071 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:57,072 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,072 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,082 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,082 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:57,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:57,083 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,083 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:57,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:57,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:57,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:57,087 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:57,088 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:57,088 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:57,088 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:57,088 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:57,088 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:57,520 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a41b0a060f1e46ce917fdf9d2ad3c85c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a41b0a060f1e46ce917fdf9d2ad3c85c 2023-02-13 12:23:57,928 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/12c97a4674ec4a77b5f168efa91e9797 2023-02-13 12:23:57,940 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/12c97a4674ec4a77b5f168efa91e9797 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797 2023-02-13 12:23:57,947 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 12:23:57,948 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 860ms, sequenceid=25, compaction requested=true 2023-02-13 12:23:57,948 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:57,948 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:57,948 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:57,949 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:57,949 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,949 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:57,949 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:57,982 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,982 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:57,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:57,983 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,983 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:57,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:57,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:57,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:57,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:57,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6058d08d[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:57,988 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:57,988 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,998 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:57,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:57,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:57,999 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:57,999 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:57,999 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:58,011 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:58,011 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:58,011 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,011 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:58,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:58,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:58,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:58,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:58,023 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:58,023 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:58,023 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:58,023 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,023 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:58,036 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,036 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:58,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,036 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:58,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:58,038 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:58,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:58,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:58,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,043 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:58,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:58,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:58,052 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:58,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:58,052 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,053 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:58,053 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:58,053 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,053 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:58,153 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:58,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:58,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:58,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:58,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,170 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:58,170 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:58,171 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:58,171 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:58,172 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,172 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,182 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:58,182 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:58,183 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:58,183 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:58,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:58,187 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:58,187 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:58,188 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:58,189 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:58,189 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:58,189 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:58,189 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:58,189 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:58,621 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e0f3a6036cbd4d058d68064d48912ded to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e0f3a6036cbd4d058d68064d48912ded 2023-02-13 12:23:59,031 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/2a3171cadba7458fa826c3710973ff2f 2023-02-13 12:23:59,044 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/2a3171cadba7458fa826c3710973ff2f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f 2023-02-13 12:23:59,050 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 12:23:59,051 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 862ms, sequenceid=29, compaction requested=true 2023-02-13 12:23:59,051 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:23:59,052 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:59,052 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:23:59,052 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:23:59,052 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,052 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:23:59,052 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:23:59,098 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,098 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:59,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:59,100 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,100 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:23:59,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:59,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:59,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:59,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:23:59,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@892cd3a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:23:59,105 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:23:59,105 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,115 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,115 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:23:59,115 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:23:59,115 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:23:59,115 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,115 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:23:59,123 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:23:59,123 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,123 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:59,124 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:23:59,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:59,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:59,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:23:59,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:23:59,132 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:23:59,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:23:59,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:23:59,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,132 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:23:59,144 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,144 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,144 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:59,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:23:59,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:23:59,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:59,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:59,146 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:23:59,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:23:59,149 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:23:59,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,160 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,161 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:59,161 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,161 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:59,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:23:59,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:59,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:59,161 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:23:59,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:23:59,162 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:23:59,161 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,161 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:23:59,162 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:23:59,162 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:23:59,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:23:59,162 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,163 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,263 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:23:59,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:23:59,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:23:59,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:23:59,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,283 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:23:59,283 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:59,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:23:59,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:23:59,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,294 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:23:59,294 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:23:59,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:23:59,294 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:23:59,294 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:23:59,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:23:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:23:59,297 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:23:59,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:23:59,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:23:59,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:23:59,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:23:59,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:23:59,298 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:23:59,299 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:23:59,299 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:23:59,721 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032b4f9824769143228cb0fa138ce97edf to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601032b4f9824769143228cb0fa138ce97edf 2023-02-13 12:24:00,131 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/32bc4418b4474d29a028bdb162096c18 2023-02-13 12:24:00,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/32bc4418b4474d29a028bdb162096c18 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18 2023-02-13 12:24:00,151 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 12:24:00,152 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 853ms, sequenceid=33, compaction requested=true 2023-02-13 12:24:00,152 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:00,152 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:00,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:00,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:00,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,153 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:24:00,153 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:24:00,190 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,190 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:00,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:00,191 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,191 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:00,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:00,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:00,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:00,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:24:00,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2843a4d9[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:00,194 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:00,194 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,220 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,221 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:00,221 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:24:00,221 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:00,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:00,235 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:24:00,235 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,235 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:00,236 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:24:00,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:00,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:00,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:00,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:00,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:00,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:00,239 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:24:00,239 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:00,239 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:00,239 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,239 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:00,248 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,248 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:00,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:00,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,248 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:00,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:00,249 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,250 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,260 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:24:00,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:00,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:00,261 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:00,262 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:00,261 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:00,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:00,363 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:00,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:00,375 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:00,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:00,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,377 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:24:00,377 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:00,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:24:00,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:00,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,424 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,424 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:00,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:00,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:00,425 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:00,425 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:00,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:00,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,428 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:24:00,428 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:00,428 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:24:00,428 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:00,429 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:24:00,430 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:00,430 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:00,430 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:00,430 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:24:00,430 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:00,860 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071cbd745e832d482aa14e304b3538f831 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509071cbd745e832d482aa14e304b3538f831 2023-02-13 12:24:01,270 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/8e1d2907c12846a3a6e533d7ccd9db13 2023-02-13 12:24:01,285 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/8e1d2907c12846a3a6e533d7ccd9db13 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13 2023-02-13 12:24:01,292 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 12:24:01,294 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 864ms, sequenceid=37, compaction requested=true 2023-02-13 12:24:01,294 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:01,294 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:01,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:01,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:01,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:24:01,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:24:01,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,332 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:01,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:01,333 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,333 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:01,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:01,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:01,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:01,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:24:01,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d40e761[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:01,339 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:01,339 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,348 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,348 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:01,348 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:24:01,348 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:01,349 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,349 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:01,356 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:24:01,356 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:01,357 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:24:01,356 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:01,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:01,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:01,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:01,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:01,363 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:01,363 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:24:01,363 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:01,363 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:01,363 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,364 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:01,373 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,373 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:01,373 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:01,374 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,375 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:01,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,385 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,385 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:01,385 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:01,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:01,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:01,385 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,386 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,386 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:24:01,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:01,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:24:01,385 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:01,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:01,386 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:01,387 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:01,387 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,387 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,387 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:01,488 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:01,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:01,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:01,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:01,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,504 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:24:01,504 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:01,504 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:24:01,504 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:01,505 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,505 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,515 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:01,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:01,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:01,515 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:01,516 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:01,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:01,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:24:01,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:01,518 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:24:01,518 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:01,520 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:24:01,520 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:01,520 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:01,520 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:01,521 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. started... 2023-02-13 12:24:01,521 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5a064ab3d6bc91c5da44588fa6275765 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:01,947 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120fd248ce326b44e96867730a90aa8a331 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fd248ce326b44e96867730a90aa8a331 2023-02-13 12:24:02,355 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/d32fbe9856d94319ae0f70af937311f7 2023-02-13 12:24:02,366 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/d32fbe9856d94319ae0f70af937311f7 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7 2023-02-13 12:24:02,372 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 12:24:02,373 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5a064ab3d6bc91c5da44588fa6275765 in 852ms, sequenceid=41, compaction requested=true 2023-02-13 12:24:02,373 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:02,373 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:02,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:02,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:02,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:24:02,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:24:02,406 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,407 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:02,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:02,407 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,407 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:02,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:02,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:02,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:02,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:24:02,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b3a94fc[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:02,411 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:02,412 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,419 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:02,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:24:02,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:02,419 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,420 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:02,427 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,427 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:24:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:02,428 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:02,428 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:24:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:02,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:02,438 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:24:02,438 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:02,438 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:02,438 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,438 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:02,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:02,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:02,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:02,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:02,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:02,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,452 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:02,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:02,465 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:02,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:24:02,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-13 12:24:02,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:02,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:02,466 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:02,467 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,467 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,467 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:02,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:02,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:02,467 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,467 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:02,567 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:02,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:02,574 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-13 12:24:02,578 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:02,579 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:02,580 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:02,580 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291642580 2023-02-13 12:24:02,581 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:02,583 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 12:24:02,583 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:02,584 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x1dbf357b to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:02,595 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37abd586, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:02,595 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:03,438 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039a7dd78ab6db42228bc633760258d1a8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601039a7dd78ab6db42228bc633760258d1a8 2023-02-13 12:24:03,440 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:03,456 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:03,457 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:35276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:03,464 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:03,465 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:53892, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:03,476 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3bd250c6d1c44c9b95b77beac6938d49 first=Optional[r5] last=Optional[r6] 2023-02-13 12:24:03,477 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3bd250c6d1c44c9b95b77beac6938d49}] 2023-02-13 12:24:03,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3bd250c6d1c44c9b95b77beac6938d49 for inclusion in 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:03,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 12:24:03,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:03,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:03,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/3bd250c6d1c44c9b95b77beac6938d49 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6mbl6942bvcrrt7qtlbau2ne7hh0jqu6tedcteqa15l8211mrnqggguc59nb99ps/family1/3bd250c6d1c44c9b95b77beac6938d49 2023-02-13 12:24:03,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6mbl6942bvcrrt7qtlbau2ne7hh0jqu6tedcteqa15l8211mrnqggguc59nb99ps/family1/3bd250c6d1c44c9b95b77beac6938d49 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_ 2023-02-13 12:24:03,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6mbl6942bvcrrt7qtlbau2ne7hh0jqu6tedcteqa15l8211mrnqggguc59nb99ps/family1/3bd250c6d1c44c9b95b77beac6938d49 into 5a064ab3d6bc91c5da44588fa6275765/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_ - updating store file list. 2023-02-13 12:24:03,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_ into 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:03,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6mbl6942bvcrrt7qtlbau2ne7hh0jqu6tedcteqa15l8211mrnqggguc59nb99ps/family1/3bd250c6d1c44c9b95b77beac6938d49 into 5a064ab3d6bc91c5da44588fa6275765/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_) 2023-02-13 12:24:03,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__6mbl6942bvcrrt7qtlbau2ne7hh0jqu6tedcteqa15l8211mrnqggguc59nb99ps/family1/3bd250c6d1c44c9b95b77beac6938d49 2023-02-13 12:24:03,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:24:03,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:24:03,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 5a064ab3d6bc91c5da44588fa6275765/family1 is initiating minor compaction 2023-02-13 12:24:03,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@55c8b144; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:03,505 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5a064ab3d6bc91c5da44588fa6275765/family1 in testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:03,505 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp, totalSize=49.7 K 2023-02-13 12:24:03,505 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 1959d955c0e344818dec2ed74b98bebe, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 2ccb64edeace4d9988993db0e3e34a0c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e351bbfa6d974fb69ca679871020560f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e1a7ff4f4b5746c1af93a235d8f6bc19, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 12c97a4674ec4a77b5f168efa91e9797, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 2a3171cadba7458fa826c3710973ff2f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 32bc4418b4474d29a028bdb162096c18, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 8e1d2907c12846a3a6e533d7ccd9db13, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting d32fbe9856d94319ae0f70af937311f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:24:03,506 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e297477169e8448ca81aad830cc64890_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:24:03,508 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:03,509 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:03,511 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e0f3a6036cbd4d058d68064d48912ded to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101e0f3a6036cbd4d058d68064d48912ded 2023-02-13 12:24:03,513 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601032b4f9824769143228cb0fa138ce97edf to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601032b4f9824769143228cb0fa138ce97edf 2023-02-13 12:24:03,514 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 12:24:03,514 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 12:24:03,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:03,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:03,562 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5a064ab3d6bc91c5da44588fa6275765#family1#compaction#22 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:24:03,984 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316327a6eb4e044dfdbfc387a98eceda74 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512316327a6eb4e044dfdbfc387a98eceda74 2023-02-13 12:24:03,985 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:04,011 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bcbf2a269cf34e82a8a15135b9853012 first=Optional[r1] last=Optional[r4] 2023-02-13 12:24:04,012 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bcbf2a269cf34e82a8a15135b9853012}] 2023-02-13 12:24:04,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bcbf2a269cf34e82a8a15135b9853012 for inclusion in 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:04,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 12:24:04,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:04,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:04,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/bcbf2a269cf34e82a8a15135b9853012 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hefjcagjf78coktq5rpu3ig92tl4m95c4qcfunhnh8gjvk7q2m0g8tcfepmojqd7/family1/bcbf2a269cf34e82a8a15135b9853012 2023-02-13 12:24:04,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hefjcagjf78coktq5rpu3ig92tl4m95c4qcfunhnh8gjvk7q2m0g8tcfepmojqd7/family1/bcbf2a269cf34e82a8a15135b9853012 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_ 2023-02-13 12:24:04,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hefjcagjf78coktq5rpu3ig92tl4m95c4qcfunhnh8gjvk7q2m0g8tcfepmojqd7/family1/bcbf2a269cf34e82a8a15135b9853012 into 5a064ab3d6bc91c5da44588fa6275765/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_ - updating store file list. 2023-02-13 12:24:04,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_ into 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:04,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hefjcagjf78coktq5rpu3ig92tl4m95c4qcfunhnh8gjvk7q2m0g8tcfepmojqd7/family1/bcbf2a269cf34e82a8a15135b9853012 into 5a064ab3d6bc91c5da44588fa6275765/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_) 2023-02-13 12:24:04,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__hefjcagjf78coktq5rpu3ig92tl4m95c4qcfunhnh8gjvk7q2m0g8tcfepmojqd7/family1/bcbf2a269cf34e82a8a15135b9853012 2023-02-13 12:24:04,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:04,029 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:04,031 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f9578b15ac4e4d77af8e06f69865733e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201f9578b15ac4e4d77af8e06f69865733e 2023-02-13 12:24:04,032 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205d9069a32a75c42d18dda8119dcb49f20 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205d9069a32a75c42d18dda8119dcb49f20 2023-02-13 12:24:04,033 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512288d7bbd62a50d42bc9703938140f7c906 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512288d7bbd62a50d42bc9703938140f7c906 2023-02-13 12:24:04,035 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a41b0a060f1e46ce917fdf9d2ad3c85c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a41b0a060f1e46ce917fdf9d2ad3c85c 2023-02-13 12:24:04,035 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 12:24:04,035 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 12:24:04,035 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:04,035 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:04,389 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/1746c9cd88e04298a4afdc8ccec91187 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1746c9cd88e04298a4afdc8ccec91187 2023-02-13 12:24:04,395 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5a064ab3d6bc91c5da44588fa6275765/family1 of 5a064ab3d6bc91c5da44588fa6275765 into 1746c9cd88e04298a4afdc8ccec91187(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:04,396 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:04,396 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676291043505; duration=0sec 2023-02-13 12:24:04,396 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:04,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_] to archive 2023-02-13 12:24:04,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:04,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1959d955c0e344818dec2ed74b98bebe 2023-02-13 12:24:04,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2ccb64edeace4d9988993db0e3e34a0c 2023-02-13 12:24:04,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e351bbfa6d974fb69ca679871020560f 2023-02-13 12:24:04,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e1a7ff4f4b5746c1af93a235d8f6bc19 2023-02-13 12:24:04,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/12c97a4674ec4a77b5f168efa91e9797 2023-02-13 12:24:04,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/2a3171cadba7458fa826c3710973ff2f 2023-02-13 12:24:04,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/32bc4418b4474d29a028bdb162096c18 2023-02-13 12:24:04,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/8e1d2907c12846a3a6e533d7ccd9db13 2023-02-13 12:24:04,483 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/d32fbe9856d94319ae0f70af937311f7 2023-02-13 12:24:04,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/e297477169e8448ca81aad830cc64890_SeqId_44_ 2023-02-13 12:24:04,490 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113047747448838d4b43a504adbb2839823a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113047747448838d4b43a504adbb2839823a 2023-02-13 12:24:04,491 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:04,517 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cef31ab58064b7682a39eac2426fe5a first=Optional[r00] last=Optional[r8] 2023-02-13 12:24:04,518 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cef31ab58064b7682a39eac2426fe5a}] 2023-02-13 12:24:04,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cef31ab58064b7682a39eac2426fe5a for inclusion in 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:04,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 12:24:04,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:04,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:04,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cef31ab58064b7682a39eac2426fe5a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vgqc5jbqht4b01oi284gmg3ke9msdko9nhmd841vmsfff97drfaj8j1jm19pk0np/family1/7cef31ab58064b7682a39eac2426fe5a 2023-02-13 12:24:04,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vgqc5jbqht4b01oi284gmg3ke9msdko9nhmd841vmsfff97drfaj8j1jm19pk0np/family1/7cef31ab58064b7682a39eac2426fe5a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_ 2023-02-13 12:24:04,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vgqc5jbqht4b01oi284gmg3ke9msdko9nhmd841vmsfff97drfaj8j1jm19pk0np/family1/7cef31ab58064b7682a39eac2426fe5a into 5a064ab3d6bc91c5da44588fa6275765/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_ - updating store file list. 2023-02-13 12:24:04,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_ into 5a064ab3d6bc91c5da44588fa6275765/family1 2023-02-13 12:24:04,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vgqc5jbqht4b01oi284gmg3ke9msdko9nhmd841vmsfff97drfaj8j1jm19pk0np/family1/7cef31ab58064b7682a39eac2426fe5a into 5a064ab3d6bc91c5da44588fa6275765/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_) 2023-02-13 12:24:04,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vgqc5jbqht4b01oi284gmg3ke9msdko9nhmd841vmsfff97drfaj8j1jm19pk0np/family1/7cef31ab58064b7682a39eac2426fe5a 2023-02-13 12:24:04,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:24:04,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:24:04,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 5a064ab3d6bc91c5da44588fa6275765/family1 is initiating minor compaction (all files) 2023-02-13 12:24:04,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@69a647e7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:24:04,540 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5a064ab3d6bc91c5da44588fa6275765/family1 in testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:24:04,540 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1746c9cd88e04298a4afdc8ccec91187, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp, totalSize=344.3 K 2023-02-13 12:24:04,541 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 49069fbec3504a41b39800890292c812, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:24:04,542 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 1746c9cd88e04298a4afdc8ccec91187, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:24:04,542 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 95c816127ccd4663973c8896e50fc1fc_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965236144 2023-02-13 12:24:04,543 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014836144 2023-02-13 12:24:04,543 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:04,543 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:04,545 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fd248ce326b44e96867730a90aa8a331 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fd248ce326b44e96867730a90aa8a331 2023-02-13 12:24:04,547 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a010dfc8ab1843c1ae0abfc67ef87551 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128a010dfc8ab1843c1ae0abfc67ef87551 2023-02-13 12:24:04,549 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130f93c8287cf204dbc97211be873ada177 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130f93c8287cf204dbc97211be873ada177 2023-02-13 12:24:04,549 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 12:24:04,549 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 12:24:04,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:04,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:04,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:24:04,550 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:24:04,550 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:04,551 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:04,553 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.9750 sec 2023-02-13 12:24:04,569 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5a064ab3d6bc91c5da44588fa6275765#family1#compaction#23 average throughput is 11.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:24:04,647 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=261 (was 250) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38854 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1dbf357b-SendThread(127.0.0.1:64508) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38812 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38774 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38966 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/cd48aa4667784da2bd10f69770ca0d98 block BP-878267689-188.40.62.62-1676291006386:blk_1073741900_1076 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: hconnection-0x73793eb-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38810 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-616355569_15 at /127.0.0.1:38974 [Receiving block BP-878267689-188.40.62.62-1676291006386:blk_1073741900_1076] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-878267689-188.40.62.62-1676291006386:blk_1073741900_1076, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38896 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x73793eb-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38886 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Thread-492 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38762 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-616355569_15 at /127.0.0.1:38822 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38860 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1dbf357b-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38736 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38952 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f119c30ed5b94c4ab7b82a794ee91273_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38704 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-878267689-188.40.62.62-1676291006386:blk_1073741900_1076 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38806 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38746 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:64508@0x1dbf357b sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/349895967.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1592944578_15 at /127.0.0.1:38876 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=612 (was 592) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=182 (was 234), ProcessCount=171 (was 171), AvailableMemoryMB=4604 (was 4667) 2023-02-13 12:24:04,659 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=261, OpenFileDescriptor=612, MaxFileDescriptor=60000, SystemLoadAverage=182, ProcessCount=171, AvailableMemoryMB=4603 2023-02-13 12:24:04,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:24:04,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:04,664 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:24:04,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-13 12:24:04,666 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:24:04,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:04,667 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:04,668 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6 empty. 2023-02-13 12:24:04,668 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:04,668 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-13 12:24:04,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:04,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:05,081 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:05,083 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0b6d53fe273287f42b700182304d00f6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:05,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:05,397 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/.tmp/family1/cd48aa4667784da2bd10f69770ca0d98 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/cd48aa4667784da2bd10f69770ca0d98 2023-02-13 12:24:05,404 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5a064ab3d6bc91c5da44588fa6275765/family1 of 5a064ab3d6bc91c5da44588fa6275765 into cd48aa4667784da2bd10f69770ca0d98(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:05,404 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5a064ab3d6bc91c5da44588fa6275765: 2023-02-13 12:24:05,405 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1676291044540; duration=0sec 2023-02-13 12:24:05,405 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:05,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1746c9cd88e04298a4afdc8ccec91187, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_] to archive 2023-02-13 12:24:05,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:05,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/49069fbec3504a41b39800890292c812 2023-02-13 12:24:05,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1746c9cd88e04298a4afdc8ccec91187 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/1746c9cd88e04298a4afdc8ccec91187 2023-02-13 12:24:05,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/95c816127ccd4663973c8896e50fc1fc_SeqId_46_ 2023-02-13 12:24:05,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/family1/13e1cb1ce10a43f6bfdf0ad271ae20ed_SeqId_49_ 2023-02-13 12:24:05,491 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:05,492 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 0b6d53fe273287f42b700182304d00f6, disabling compactions & flushes 2023-02-13 12:24:05,492 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,492 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,492 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. after waiting 0 ms 2023-02-13 12:24:05,492 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,492 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,492 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 0b6d53fe273287f42b700182304d00f6: Waiting for close lock at 1676291045492Disabling compacts and flushes for region at 1676291045492Disabling writes for close at 1676291045492Writing region close event to WAL at 1676291045492Closed at 1676291045492 2023-02-13 12:24:05,497 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:24:05,499 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291045498"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291045498"}]},"ts":"1676291045498"} 2023-02-13 12:24:05,502 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:24:05,503 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:24:05,503 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291045503"}]},"ts":"1676291045503"} 2023-02-13 12:24:05,505 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 12:24:05,654 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0b6d53fe273287f42b700182304d00f6, ASSIGN}] 2023-02-13 12:24:05,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0b6d53fe273287f42b700182304d00f6, ASSIGN 2023-02-13 12:24:05,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0b6d53fe273287f42b700182304d00f6, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:05,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:05,814 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=0b6d53fe273287f42b700182304d00f6, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:05,815 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291045814"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291045814"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291045814"}]},"ts":"1676291045814"} 2023-02-13 12:24:05,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 0b6d53fe273287f42b700182304d00f6, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:05,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0b6d53fe273287f42b700182304d00f6, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:05,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:05,975 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,977 INFO [StoreOpener-0b6d53fe273287f42b700182304d00f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,979 DEBUG [StoreOpener-0b6d53fe273287f42b700182304d00f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:05,979 DEBUG [StoreOpener-0b6d53fe273287f42b700182304d00f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:05,979 INFO [StoreOpener-0b6d53fe273287f42b700182304d00f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0b6d53fe273287f42b700182304d00f6 columnFamilyName family1 2023-02-13 12:24:05,981 INFO [StoreOpener-0b6d53fe273287f42b700182304d00f6-1] regionserver.HStore(310): Store=0b6d53fe273287f42b700182304d00f6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:05,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:24:05,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:05,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 0b6d53fe273287f42b700182304d00f6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72005529, jitterRate=0.07296599447727203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:05,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 0b6d53fe273287f42b700182304d00f6: Running coprocessor pre-open hook at 1676291045976Writing region info on filesystem at 1676291045976Initializing all the Stores at 1676291045977 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291045977Cleaning up temporary data from old regions at 1676291045984 (+7 ms)Cleaning up detritus from prior splits at 1676291045985 (+1 ms)Running coprocessor post-open hooks at 1676291045991 (+6 ms)Region opened successfully at 1676291045991 2023-02-13 12:24:05,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6., pid=25, masterSystemTime=1676291045972 2023-02-13 12:24:05,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:05,994 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=0b6d53fe273287f42b700182304d00f6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:05,994 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291045994"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291045994"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291045994"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291045994"}]},"ts":"1676291045994"} 2023-02-13 12:24:05,998 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-13 12:24:05,998 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 0b6d53fe273287f42b700182304d00f6, server=jenkins-hbase16.apache.org,38471,1676291011293 in 177 msec 2023-02-13 12:24:06,000 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-13 12:24:06,000 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=0b6d53fe273287f42b700182304d00f6, ASSIGN in 344 msec 2023-02-13 12:24:06,001 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:24:06,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291046001"}]},"ts":"1676291046001"} 2023-02-13 12:24:06,002 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 12:24:06,011 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:24:06,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.3500 sec 2023-02-13 12:24:06,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-13 12:24:06,775 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-13 12:24:06,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:06,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,817 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:06,817 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:06,817 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:06,817 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:06,817 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,817 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,827 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:06,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:06,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:06,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:06,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:06,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:06,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,828 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:06,828 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:06,828 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:06,828 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:06,829 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:06,829 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:06,829 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:06,829 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:06,830 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:24:06,858 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113082c6b5e02f1744b5b883021651741638 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015113082c6b5e02f1744b5b883021651741638 2023-02-13 12:24:07,264 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/d0fee9a87a0b4acbb2a03f8ffd026b33 2023-02-13 12:24:07,276 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/d0fee9a87a0b4acbb2a03f8ffd026b33 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33 2023-02-13 12:24:07,281 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 12:24:07,282 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 453ms, sequenceid=5, compaction requested=false 2023-02-13 12:24:07,282 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:07,283 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:07,283 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:07,283 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:07,283 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:07,283 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,283 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:07,283 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:07,315 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,315 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:07,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:07,316 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,316 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:07,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:07,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:07,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:07,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:07,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5df71bc[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:07,320 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:07,320 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,332 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:07,332 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:07,332 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:07,332 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,333 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:07,344 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:07,344 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:07,344 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:07,344 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:07,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:07,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:07,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:07,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:07,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:07,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:07,355 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:07,355 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:07,355 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:07,355 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,355 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:07,364 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,364 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:07,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,365 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:07,365 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:07,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:07,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:07,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:07,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:07,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,377 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:07,377 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:07,378 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:07,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:07,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,378 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:07,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:07,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:07,478 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:07,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:07,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:07,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:07,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,495 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:07,495 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:07,495 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:07,495 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:07,496 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,497 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,507 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:07,507 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:07,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:07,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:07,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:07,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:07,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,511 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:07,512 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:07,512 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:07,512 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:07,512 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:07,512 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:07,513 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:07,513 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:07,513 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:07,513 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:24:07,940 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128eed70c04aa32420da3b5bde1fe454de0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128eed70c04aa32420da3b5bde1fe454de0 2023-02-13 12:24:08,348 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/a281098f63cd4667b9ba6de43e7918b5 2023-02-13 12:24:08,361 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/a281098f63cd4667b9ba6de43e7918b5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5 2023-02-13 12:24:08,369 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:24:08,370 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 857ms, sequenceid=9, compaction requested=false 2023-02-13 12:24:08,371 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:08,371 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:08,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:08,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:08,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:08,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:08,406 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,406 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:08,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:08,407 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,407 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:08,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:08,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:08,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:08,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:08,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@300f78b2[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:08,411 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:08,411 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,418 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:08,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:08,419 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:08,419 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,419 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:08,427 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:08,427 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:08,427 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,427 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:08,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:08,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:08,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:08,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:08,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:08,435 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:08,435 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:08,435 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:08,435 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,436 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:08,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:08,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:08,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,448 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:08,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:08,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:08,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:08,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:08,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,454 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:08,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:08,465 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,465 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:08,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,465 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:08,466 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:08,466 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,466 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:08,466 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:08,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:08,566 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:08,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:08,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:08,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:08,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,582 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:08,582 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:08,582 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:08,582 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:08,583 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,583 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,593 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,593 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:08,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:08,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:08,594 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:08,594 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:08,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:08,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,595 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:08,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:08,595 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:08,595 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:08,595 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:08,595 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:08,595 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:08,595 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:08,595 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:08,596 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:08,620 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:08,620 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1dbf357b to 127.0.0.1:64508 2023-02-13 12:24:08,620 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:08,621 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:08,621 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5af77c43 to 127.0.0.1:64508 2023-02-13 12:24:08,621 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:08,622 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:08,622 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1335b40e to 127.0.0.1:64508 2023-02-13 12:24:08,622 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:09,035 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011cae6ea5afaf4270a4350f46bd34d235 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512011cae6ea5afaf4270a4350f46bd34d235 2023-02-13 12:24:09,444 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/b33ade23537d46a9aff69a1f23ed5e54 2023-02-13 12:24:09,457 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/b33ade23537d46a9aff69a1f23ed5e54 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54 2023-02-13 12:24:09,463 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 12:24:09,464 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 868ms, sequenceid=13, compaction requested=true 2023-02-13 12:24:09,464 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:09,465 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:09,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:09,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:09,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:09,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:09,498 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:09,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:09,499 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:09,499 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:09,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:09,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:09,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:09,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@acefa3d[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:09,502 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:09,503 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,510 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,510 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:09,510 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:09,511 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:09,511 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,511 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:09,518 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:09,519 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:09,519 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,519 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:09,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:09,527 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:09,527 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:09,527 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:09,527 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,527 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:09,566 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,566 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:09,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:09,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,566 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:09,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:09,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:09,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:09,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:09,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,570 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:09,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,585 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,585 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,585 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,585 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,585 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:09,585 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:09,586 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:09,586 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:09,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:09,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:09,586 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:09,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:09,586 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,587 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:09,587 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:09,587 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:09,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:09,687 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:09,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:09,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:09,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:09,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,699 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:09,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:09,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:09,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:09,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,710 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,710 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:09,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:09,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:09,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:09,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:09,712 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:09,712 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:09,712 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:09,712 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:09,712 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:09,712 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:09,713 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:09,713 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:10,138 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c90f0488c7cd4bbf9ff787f58d937742 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205c90f0488c7cd4bbf9ff787f58d937742 2023-02-13 12:24:10,546 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/59602630458d4f06ba0979f17ed84680 2023-02-13 12:24:10,552 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/59602630458d4f06ba0979f17ed84680 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680 2023-02-13 12:24:10,558 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 12:24:10,559 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 846ms, sequenceid=17, compaction requested=true 2023-02-13 12:24:10,559 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:10,559 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:10,559 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:10,559 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:10,559 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:10,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:10,577 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,577 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:10,577 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,577 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:10,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:10,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@459c12ec[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:10,579 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:10,580 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,590 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,590 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:10,590 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:10,590 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:10,590 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,590 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:10,598 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,598 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:10,598 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:10,598 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:10,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:10,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:10,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:10,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:10,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:10,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:10,606 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:10,606 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:10,607 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:10,607 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,607 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:10,619 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,619 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:10,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:10,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,619 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:10,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:10,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:10,620 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:10,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,624 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:10,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,635 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,635 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:10,635 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:10,635 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:10,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:10,635 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:10,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:10,635 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,636 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:10,636 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:10,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:10,636 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,636 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,636 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,637 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,736 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:10,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:10,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:10,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:10,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,752 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:10,752 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:10,752 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:10,752 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:10,753 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,753 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,760 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:10,760 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:10,760 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:10,760 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:10,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,761 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:10,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,761 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:10,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:10,762 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:10,762 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:10,762 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:10,762 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:10,762 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:10,762 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:10,762 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:10,762 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:11,181 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e73b182cca3b4c03b5ce9c2e4190499b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e73b182cca3b4c03b5ce9c2e4190499b 2023-02-13 12:24:11,512 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:24:11,589 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/6c5d30fd56ae447fbdee55c167023a2c 2023-02-13 12:24:11,596 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/6c5d30fd56ae447fbdee55c167023a2c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c 2023-02-13 12:24:11,602 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 12:24:11,603 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 841ms, sequenceid=21, compaction requested=true 2023-02-13 12:24:11,603 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:11,603 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:11,603 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:11,603 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:11,603 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,603 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:11,603 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:11,640 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,640 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:11,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,641 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:11,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:11,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:11,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:11,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:11,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b30c972[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:11,646 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:11,646 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,656 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:11,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:11,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:11,657 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,658 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:11,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:11,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:11,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:11,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:11,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:11,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:11,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:11,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:11,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:11,677 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:11,677 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:11,677 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:11,677 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,677 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:11,685 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,685 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:11,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:11,685 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:11,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:11,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:11,686 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:11,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,688 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,698 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:11,698 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:11,698 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,698 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:11,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:11,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:11,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:11,698 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:11,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:11,699 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:11,800 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:11,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:11,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:11,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:11,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,812 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:11,812 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:11,812 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:11,812 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:11,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,813 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,823 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:11,823 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:11,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:11,824 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:11,824 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:11,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:11,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:11,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:11,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:11,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:11,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:11,826 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:11,826 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:11,826 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:12,254 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314bf410a844eb4c03a2f063d4de481424 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314bf410a844eb4c03a2f063d4de481424 2023-02-13 12:24:12,662 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/16cfb84e9bcc430081e42300f514a380 2023-02-13 12:24:12,671 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/16cfb84e9bcc430081e42300f514a380 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380 2023-02-13 12:24:12,678 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 12:24:12,679 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 853ms, sequenceid=25, compaction requested=true 2023-02-13 12:24:12,679 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:12,679 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:12,679 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:12,679 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:12,679 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,679 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:12,679 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:12,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:12,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:12,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:12,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:12,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:12,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:12,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:12,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7031ec4e[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:12,720 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:12,720 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,731 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,731 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,731 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:12,731 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:12,731 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:12,731 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:12,757 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:12,757 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:12,757 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:12,757 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:12,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:12,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:12,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:12,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:12,763 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:12,764 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:12,764 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:12,764 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,764 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:12,777 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,777 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:12,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,777 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:12,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:12,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:12,778 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:12,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:12,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,781 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:12,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,793 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,793 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:12,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:12,794 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:12,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:12,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:12,794 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:12,795 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:12,795 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:12,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:12,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:12,896 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:12,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:12,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:12,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:12,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,911 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:12,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:12,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:12,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:12,912 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,912 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,918 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:12,918 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:12,919 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:12,919 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:12,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,922 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:12,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:12,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:12,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:12,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:12,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:12,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:12,923 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:12,923 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:12,924 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:13,348 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601017cbede1d8bbb43929f7c39f590ed3101 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017cbede1d8bbb43929f7c39f590ed3101 2023-02-13 12:24:13,357 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/7198c3ea5d154c5bac14e588778e1c5b 2023-02-13 12:24:13,365 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/7198c3ea5d154c5bac14e588778e1c5b as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b 2023-02-13 12:24:13,372 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 12:24:13,373 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 449ms, sequenceid=29, compaction requested=true 2023-02-13 12:24:13,373 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:13,373 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:13,373 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:13,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:13,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:13,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:13,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,389 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:13,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:13,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:13,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:13,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:13,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:13,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:13,392 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d53ff67[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:13,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:13,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,402 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,402 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:13,402 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:13,402 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:13,402 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,402 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:13,410 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:13,410 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,410 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:13,410 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:13,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:13,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:13,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:13,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:13,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:13,417 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:13,417 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:13,417 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:13,417 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,417 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:13,427 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,427 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:13,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,427 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:13,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:13,428 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:13,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,432 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,443 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,443 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:13,443 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:13,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:13,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:13,443 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:13,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:13,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:13,443 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:13,443 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:13,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,444 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:13,544 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:13,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:13,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:13,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:13,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,557 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:13,557 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:13,557 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:13,557 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:13,558 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,558 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,568 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,568 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:13,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:13,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:13,569 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:13,569 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:13,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:13,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:13,570 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:13,570 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:13,570 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:13,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:13,571 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:13,571 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:13,571 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:13,571 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:13,997 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d2154efde37d4ba3b579613f6044b639 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d2154efde37d4ba3b579613f6044b639 2023-02-13 12:24:14,408 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/19194f27903d46079e7d5b8e3a949b76 2023-02-13 12:24:14,423 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/19194f27903d46079e7d5b8e3a949b76 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76 2023-02-13 12:24:14,430 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 12:24:14,431 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 860ms, sequenceid=33, compaction requested=true 2023-02-13 12:24:14,431 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:14,431 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:14,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:14,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:14,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:14,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:14,473 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,473 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:14,474 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,474 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:14,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:14,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:14,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@287d9753[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:14,478 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:14,478 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,490 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,490 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:14,490 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:14,490 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:14,491 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,491 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:14,502 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:14,502 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:14,502 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:14,502 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:14,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:14,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:14,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:14,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:14,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:14,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:14,514 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:14,514 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:14,514 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:14,514 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,514 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:14,527 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,527 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:14,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,527 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:14,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:14,529 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:14,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,532 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,543 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:14,543 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:14,543 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:14,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:14,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:14,543 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:14,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:14,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:14,544 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:14,644 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:14,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:14,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:14,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:14,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,661 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:14,661 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:14,661 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:14,661 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:14,662 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,662 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,673 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:14,673 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:14,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:14,673 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:14,673 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:14,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:14,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,675 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,675 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:14,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:14,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:14,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:14,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:14,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:14,676 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:14,676 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:14,676 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:14,676 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:15,102 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ff360797c0d3441f9bf9ef13df64b7f3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907ff360797c0d3441f9bf9ef13df64b7f3 2023-02-13 12:24:15,511 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/938e6d240297418c9791bfbaba712faf 2023-02-13 12:24:15,521 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/938e6d240297418c9791bfbaba712faf as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf 2023-02-13 12:24:15,526 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 12:24:15,528 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 852ms, sequenceid=37, compaction requested=true 2023-02-13 12:24:15,528 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:15,528 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:15,528 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:15,528 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:15,528 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,528 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:15,528 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:15,591 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,591 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:15,592 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,592 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:15,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:15,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15bba077[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:15,595 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:15,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,610 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:15,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:15,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:15,610 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,611 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:15,618 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:15,619 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:15,619 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:15,618 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:15,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:15,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:15,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:15,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:15,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:15,627 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:15,627 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:15,627 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:15,627 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,627 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:15,639 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,639 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:15,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,640 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:15,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:15,641 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,643 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:15,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,668 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:15,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,669 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:15,669 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:15,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:15,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:15,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:15,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:15,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:15,670 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:15,670 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:15,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,670 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,671 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,671 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:15,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:15,770 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:15,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:15,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:15,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:15,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,787 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:15,787 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:15,787 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:15,787 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:15,788 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,788 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,798 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,798 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:15,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:15,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:15,798 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:15,798 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:15,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:15,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,801 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,801 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:15,801 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:15,801 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:15,802 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:15,802 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:15,802 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:15,802 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:15,802 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:15,802 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. started... 2023-02-13 12:24:15,803 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0b6d53fe273287f42b700182304d00f6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:16,232 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e1ba6b971aa84dc995eacac669bdd21f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120e1ba6b971aa84dc995eacac669bdd21f 2023-02-13 12:24:16,640 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/142e8bcb84874422a279ba2ee5a8d0ca 2023-02-13 12:24:16,651 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/142e8bcb84874422a279ba2ee5a8d0ca as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca 2023-02-13 12:24:16,656 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 12:24:16,657 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0b6d53fe273287f42b700182304d00f6 in 855ms, sequenceid=41, compaction requested=true 2023-02-13 12:24:16,657 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:16,657 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:16,657 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:16,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,689 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:16,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:16,690 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,690 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:16,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:16,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:16,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:16,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:16,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a976d12[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:16,692 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:16,692 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,701 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,702 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:16,702 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,702 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:16,702 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:16,702 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:16,710 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:16,710 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,710 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:16,710 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:16,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:16,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:16,712 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:16,712 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:16,712 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:16,712 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,712 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:16,723 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,723 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:16,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,723 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:16,723 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:16,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:16,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,724 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:16,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:16,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:16,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:16,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-13 12:24:16,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:16,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:16,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,735 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:16,736 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:16,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:16,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:16,836 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:16,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:16,838 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-13 12:24:16,840 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:16,841 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:16,842 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291656842 2023-02-13 12:24:16,842 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:16,845 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:24:16,847 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 12:24:16,847 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:16,847 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x7844ff08 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:16,860 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43d79320, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:16,861 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,726 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b398e5af7d6b434fa968ca8d633255b7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103b398e5af7d6b434fa968ca8d633255b7 2023-02-13 12:24:17,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:17,742 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:17,743 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60828, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:17,750 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:17,753 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60076, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:17,762 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad7786eb7f4940aa9c840865f694e4bd first=Optional[r3] last=Optional[r6] 2023-02-13 12:24:17,763 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad7786eb7f4940aa9c840865f694e4bd}] 2023-02-13 12:24:17,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad7786eb7f4940aa9c840865f694e4bd for inclusion in 0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:17,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 12:24:17,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:17,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:17,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/ad7786eb7f4940aa9c840865f694e4bd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__tc0gd0vgfgj9hthoanfrdh8ps3k2ihdopcbr9b9vlll922879jek6spib0cg93p/family1/ad7786eb7f4940aa9c840865f694e4bd 2023-02-13 12:24:17,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__tc0gd0vgfgj9hthoanfrdh8ps3k2ihdopcbr9b9vlll922879jek6spib0cg93p/family1/ad7786eb7f4940aa9c840865f694e4bd as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_ 2023-02-13 12:24:17,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__tc0gd0vgfgj9hthoanfrdh8ps3k2ihdopcbr9b9vlll922879jek6spib0cg93p/family1/ad7786eb7f4940aa9c840865f694e4bd into 0b6d53fe273287f42b700182304d00f6/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_ - updating store file list. 2023-02-13 12:24:17,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_ into 0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:17,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__tc0gd0vgfgj9hthoanfrdh8ps3k2ihdopcbr9b9vlll922879jek6spib0cg93p/family1/ad7786eb7f4940aa9c840865f694e4bd into 0b6d53fe273287f42b700182304d00f6/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_) 2023-02-13 12:24:17,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__tc0gd0vgfgj9hthoanfrdh8ps3k2ihdopcbr9b9vlll922879jek6spib0cg93p/family1/ad7786eb7f4940aa9c840865f694e4bd 2023-02-13 12:24:17,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:24:17,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:24:17,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 0b6d53fe273287f42b700182304d00f6/family1 is initiating minor compaction 2023-02-13 12:24:17,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4e71956d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:17,784 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0b6d53fe273287f42b700182304d00f6/family1 in testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:17,784 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp, totalSize=50.0 K 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting a281098f63cd4667b9ba6de43e7918b5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting b33ade23537d46a9aff69a1f23ed5e54, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 59602630458d4f06ba0979f17ed84680, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 6c5d30fd56ae447fbdee55c167023a2c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 16cfb84e9bcc430081e42300f514a380, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 7198c3ea5d154c5bac14e588778e1c5b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 19194f27903d46079e7d5b8e3a949b76, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:24:17,784 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 938e6d240297418c9791bfbaba712faf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:24:17,785 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 142e8bcb84874422a279ba2ee5a8d0ca, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:24:17,785 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting b022354193e24025895f40ff06fb74bf_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:24:17,786 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:17,787 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:17,788 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e73b182cca3b4c03b5ce9c2e4190499b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228e73b182cca3b4c03b5ce9c2e4190499b 2023-02-13 12:24:17,789 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314bf410a844eb4c03a2f063d4de481424 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512314bf410a844eb4c03a2f063d4de481424 2023-02-13 12:24:17,790 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017cbede1d8bbb43929f7c39f590ed3101 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601017cbede1d8bbb43929f7c39f590ed3101 2023-02-13 12:24:17,791 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d2154efde37d4ba3b579613f6044b639 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103d2154efde37d4ba3b579613f6044b639 2023-02-13 12:24:17,791 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 12:24:17,791 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,792 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:17,808 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0b6d53fe273287f42b700182304d00f6#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:24:18,631 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052e4151d9b3d545179fc7b413f4d97338 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512052e4151d9b3d545179fc7b413f4d97338 2023-02-13 12:24:18,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:18,632 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/be31e1d526d54d76b4b8d471b8db08bb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/be31e1d526d54d76b4b8d471b8db08bb 2023-02-13 12:24:18,639 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 0b6d53fe273287f42b700182304d00f6/family1 of 0b6d53fe273287f42b700182304d00f6 into be31e1d526d54d76b4b8d471b8db08bb(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:18,639 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:18,640 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676291057783; duration=0sec 2023-02-13 12:24:18,640 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:18,658 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9be4ecafbb3416b9176542f04ae5ef6 first=Optional[r1] last=Optional[r2] 2023-02-13 12:24:18,659 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9be4ecafbb3416b9176542f04ae5ef6}] 2023-02-13 12:24:18,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9be4ecafbb3416b9176542f04ae5ef6 for inclusion in 0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:18,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_] to archive 2023-02-13 12:24:18,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:18,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 12:24:18,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:18,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:18,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e9be4ecafbb3416b9176542f04ae5ef6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9mvo89p69f6iupaejsk0mngpqm2qc5vi9r1d0gsb4l4ibqupnrmtnmvju4ndteb3/family1/e9be4ecafbb3416b9176542f04ae5ef6 2023-02-13 12:24:18,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/a281098f63cd4667b9ba6de43e7918b5 2023-02-13 12:24:18,680 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b33ade23537d46a9aff69a1f23ed5e54 2023-02-13 12:24:18,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9mvo89p69f6iupaejsk0mngpqm2qc5vi9r1d0gsb4l4ibqupnrmtnmvju4ndteb3/family1/e9be4ecafbb3416b9176542f04ae5ef6 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_ 2023-02-13 12:24:18,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9mvo89p69f6iupaejsk0mngpqm2qc5vi9r1d0gsb4l4ibqupnrmtnmvju4ndteb3/family1/e9be4ecafbb3416b9176542f04ae5ef6 into 0b6d53fe273287f42b700182304d00f6/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_ - updating store file list. 2023-02-13 12:24:18,681 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/59602630458d4f06ba0979f17ed84680 2023-02-13 12:24:18,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/6c5d30fd56ae447fbdee55c167023a2c 2023-02-13 12:24:18,684 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/16cfb84e9bcc430081e42300f514a380 2023-02-13 12:24:18,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/7198c3ea5d154c5bac14e588778e1c5b 2023-02-13 12:24:18,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_ into 0b6d53fe273287f42b700182304d00f6/family1 2023-02-13 12:24:18,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9mvo89p69f6iupaejsk0mngpqm2qc5vi9r1d0gsb4l4ibqupnrmtnmvju4ndteb3/family1/e9be4ecafbb3416b9176542f04ae5ef6 into 0b6d53fe273287f42b700182304d00f6/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_) 2023-02-13 12:24:18,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithWeeklyPolicy__9mvo89p69f6iupaejsk0mngpqm2qc5vi9r1d0gsb4l4ibqupnrmtnmvju4ndteb3/family1/e9be4ecafbb3416b9176542f04ae5ef6 2023-02-13 12:24:18,688 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/19194f27903d46079e7d5b8e3a949b76 2023-02-13 12:24:18,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:24:18,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:24:18,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 0b6d53fe273287f42b700182304d00f6/family1 is initiating minor compaction (all files) 2023-02-13 12:24:18,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@f6a8567; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:18,689 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0b6d53fe273287f42b700182304d00f6/family1 in testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:24:18,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/938e6d240297418c9791bfbaba712faf 2023-02-13 12:24:18,690 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/be31e1d526d54d76b4b8d471b8db08bb, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp, totalSize=176.8 K 2023-02-13 12:24:18,690 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting d0fee9a87a0b4acbb2a03f8ffd026b33, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:24:18,691 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting be31e1d526d54d76b4b8d471b8db08bb, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:24:18,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/142e8bcb84874422a279ba2ee5a8d0ca 2023-02-13 12:24:18,692 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 430fbe74b1b9451583b168d46f374df7_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965236144 2023-02-13 12:24:18,692 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:18,692 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:18,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/b022354193e24025895f40ff06fb74bf_SeqId_44_ 2023-02-13 12:24:18,694 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512011cae6ea5afaf4270a4350f46bd34d235 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512011cae6ea5afaf4270a4350f46bd34d235 2023-02-13 12:24:18,695 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205c90f0488c7cd4bbf9ff787f58d937742 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205c90f0488c7cd4bbf9ff787f58d937742 2023-02-13 12:24:18,695 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-13 12:24:18,695 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 12:24:18,696 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:18,696 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:18,696 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:18,697 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:18,699 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8580 sec 2023-02-13 12:24:18,709 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0b6d53fe273287f42b700182304d00f6#family1#compaction#35 average throughput is 25.77 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:24:18,785 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=250 (was 261), OpenFileDescriptor=581 (was 612), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=170 (was 182), ProcessCount=169 (was 171), AvailableMemoryMB=6087 (was 4603) - AvailableMemoryMB LEAK? - 2023-02-13 12:24:18,795 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=250, OpenFileDescriptor=581, MaxFileDescriptor=60000, SystemLoadAverage=170, ProcessCount=169, AvailableMemoryMB=6087 2023-02-13 12:24:18,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 12:24:18,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:18,800 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:24:18,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-13 12:24:18,801 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:24:18,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:18,803 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173 2023-02-13 12:24:18,803 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:18,803 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08 empty. 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a empty. 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173 empty. 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:18,804 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173 2023-02-13 12:24:18,804 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-13 12:24:18,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:19,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:19,215 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:19,218 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 956d36d3510f1e7466a907210667b173, NAME => 'testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:19,218 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 4c42cd6a08dfd848a6e7ca94fad32a2a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:19,219 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 976a16fc4c9a9b309025ba2a32e7ab08, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:19,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:19,638 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:19,639 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 4c42cd6a08dfd848a6e7ca94fad32a2a, disabling compactions & flushes 2023-02-13 12:24:19,639 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:19,639 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:19,639 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:19,640 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 956d36d3510f1e7466a907210667b173, disabling compactions & flushes 2023-02-13 12:24:19,640 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. after waiting 0 ms 2023-02-13 12:24:19,640 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:19,640 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:19,640 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:19,641 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:19,640 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 976a16fc4c9a9b309025ba2a32e7ab08, disabling compactions & flushes 2023-02-13 12:24:19,641 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: Waiting for close lock at 1676291059639Disabling compacts and flushes for region at 1676291059639Disabling writes for close at 1676291059640 (+1 ms)Writing region close event to WAL at 1676291059641 (+1 ms)Closed at 1676291059641 2023-02-13 12:24:19,641 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. after waiting 0 ms 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:19,642 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:19,641 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. after waiting 0 ms 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:19,642 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 956d36d3510f1e7466a907210667b173: Waiting for close lock at 1676291059640Disabling compacts and flushes for region at 1676291059640Disabling writes for close at 1676291059642 (+2 ms)Writing region close event to WAL at 1676291059642Closed at 1676291059642 2023-02-13 12:24:19,643 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:19,643 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 976a16fc4c9a9b309025ba2a32e7ab08: Waiting for close lock at 1676291059640Disabling compacts and flushes for region at 1676291059640Disabling writes for close at 1676291059642 (+2 ms)Writing region close event to WAL at 1676291059643 (+1 ms)Closed at 1676291059643 2023-02-13 12:24:19,646 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:24:19,648 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291059648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291059648"}]},"ts":"1676291059648"} 2023-02-13 12:24:19,648 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291059648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291059648"}]},"ts":"1676291059648"} 2023-02-13 12:24:19,648 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676291059648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291059648"}]},"ts":"1676291059648"} 2023-02-13 12:24:19,651 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 12:24:19,653 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:24:19,653 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291059653"}]},"ts":"1676291059653"} 2023-02-13 12:24:19,655 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-13 12:24:19,707 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=956d36d3510f1e7466a907210667b173, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=976a16fc4c9a9b309025ba2a32e7ab08, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4c42cd6a08dfd848a6e7ca94fad32a2a, ASSIGN}] 2023-02-13 12:24:19,711 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4c42cd6a08dfd848a6e7ca94fad32a2a, ASSIGN 2023-02-13 12:24:19,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=976a16fc4c9a9b309025ba2a32e7ab08, ASSIGN 2023-02-13 12:24:19,712 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=956d36d3510f1e7466a907210667b173, ASSIGN 2023-02-13 12:24:19,713 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4c42cd6a08dfd848a6e7ca94fad32a2a, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:19,713 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=956d36d3510f1e7466a907210667b173, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:19,713 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=976a16fc4c9a9b309025ba2a32e7ab08, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:19,867 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=4c42cd6a08dfd848a6e7ca94fad32a2a, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:19,867 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=956d36d3510f1e7466a907210667b173, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:19,867 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=976a16fc4c9a9b309025ba2a32e7ab08, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:19,867 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291059867"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291059867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291059867"}]},"ts":"1676291059867"} 2023-02-13 12:24:19,867 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291059867"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291059867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291059867"}]},"ts":"1676291059867"} 2023-02-13 12:24:19,868 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676291059867"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291059867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291059867"}]},"ts":"1676291059867"} 2023-02-13 12:24:19,871 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 4c42cd6a08dfd848a6e7ca94fad32a2a, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:19,872 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 956d36d3510f1e7466a907210667b173, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:19,873 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 976a16fc4c9a9b309025ba2a32e7ab08, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:19,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:19,946 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/.tmp/family1/ef0cec179674408890211200456c950f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/ef0cec179674408890211200456c950f 2023-02-13 12:24:19,955 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 0b6d53fe273287f42b700182304d00f6/family1 of 0b6d53fe273287f42b700182304d00f6 into ef0cec179674408890211200456c950f(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:24:19,955 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0b6d53fe273287f42b700182304d00f6: 2023-02-13 12:24:19,955 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1676291058689; duration=1sec 2023-02-13 12:24:19,955 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:19,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/be31e1d526d54d76b4b8d471b8db08bb, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_] to archive 2023-02-13 12:24:19,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:19,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/d0fee9a87a0b4acbb2a03f8ffd026b33 2023-02-13 12:24:19,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/be31e1d526d54d76b4b8d471b8db08bb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/be31e1d526d54d76b4b8d471b8db08bb 2023-02-13 12:24:19,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/family1/430fbe74b1b9451583b168d46f374df7_SeqId_47_ 2023-02-13 12:24:20,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:20,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4c42cd6a08dfd848a6e7ca94fad32a2a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 12:24:20,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:20,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,029 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,031 DEBUG [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1 2023-02-13 12:24:20,031 DEBUG [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1 2023-02-13 12:24:20,032 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4c42cd6a08dfd848a6e7ca94fad32a2a columnFamilyName family1 2023-02-13 12:24:20,034 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] regionserver.HStore(310): Store=4c42cd6a08dfd848a6e7ca94fad32a2a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,034 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,036 DEBUG [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family2 2023-02-13 12:24:20,036 DEBUG [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family2 2023-02-13 12:24:20,036 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4c42cd6a08dfd848a6e7ca94fad32a2a columnFamilyName family2 2023-02-13 12:24:20,038 INFO [StoreOpener-4c42cd6a08dfd848a6e7ca94fad32a2a-1] regionserver.HStore(310): Store=4c42cd6a08dfd848a6e7ca94fad32a2a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:24:20,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:24:20,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:20,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 4c42cd6a08dfd848a6e7ca94fad32a2a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72104501, jitterRate=0.07444079220294952}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:24:20,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: Running coprocessor pre-open hook at 1676291060028Writing region info on filesystem at 1676291060028Initializing all the Stores at 1676291060029 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060029Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060029Cleaning up temporary data from old regions at 1676291060041 (+12 ms)Cleaning up detritus from prior splits at 1676291060041Running coprocessor post-open hooks at 1676291060047 (+6 ms)Region opened successfully at 1676291060047 2023-02-13 12:24:20,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a., pid=31, masterSystemTime=1676291060024 2023-02-13 12:24:20,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:20,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:20,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:20,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 956d36d3510f1e7466a907210667b173, NAME => 'testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 12:24:20,051 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=4c42cd6a08dfd848a6e7ca94fad32a2a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:20,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:20,051 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291060051"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291060051"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291060051"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291060051"}]},"ts":"1676291060051"} 2023-02-13 12:24:20,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,053 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,055 DEBUG [StoreOpener-956d36d3510f1e7466a907210667b173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/family1 2023-02-13 12:24:20,055 DEBUG [StoreOpener-956d36d3510f1e7466a907210667b173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/family1 2023-02-13 12:24:20,056 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 956d36d3510f1e7466a907210667b173 columnFamilyName family1 2023-02-13 12:24:20,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-02-13 12:24:20,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 4c42cd6a08dfd848a6e7ca94fad32a2a, server=jenkins-hbase16.apache.org,38471,1676291011293 in 182 msec 2023-02-13 12:24:20,057 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=4c42cd6a08dfd848a6e7ca94fad32a2a, ASSIGN in 349 msec 2023-02-13 12:24:20,058 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] regionserver.HStore(310): Store=956d36d3510f1e7466a907210667b173/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,058 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,059 DEBUG [StoreOpener-956d36d3510f1e7466a907210667b173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/family2 2023-02-13 12:24:20,060 DEBUG [StoreOpener-956d36d3510f1e7466a907210667b173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/family2 2023-02-13 12:24:20,060 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 956d36d3510f1e7466a907210667b173 columnFamilyName family2 2023-02-13 12:24:20,061 INFO [StoreOpener-956d36d3510f1e7466a907210667b173-1] regionserver.HStore(310): Store=956d36d3510f1e7466a907210667b173/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:24:20,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 956d36d3510f1e7466a907210667b173 2023-02-13 12:24:20,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:20,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 956d36d3510f1e7466a907210667b173; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66895732, jitterRate=-0.0031759142875671387}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:24:20,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 956d36d3510f1e7466a907210667b173: Running coprocessor pre-open hook at 1676291060052Writing region info on filesystem at 1676291060052Initializing all the Stores at 1676291060053 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060053Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060053Cleaning up temporary data from old regions at 1676291060063 (+10 ms)Cleaning up detritus from prior splits at 1676291060063Running coprocessor post-open hooks at 1676291060067 (+4 ms)Region opened successfully at 1676291060067 2023-02-13 12:24:20,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173., pid=32, masterSystemTime=1676291060024 2023-02-13 12:24:20,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:20,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:20,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:20,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 976a16fc4c9a9b309025ba2a32e7ab08, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 12:24:20,070 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=956d36d3510f1e7466a907210667b173, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:20,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:20,070 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676291060070"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291060070"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291060070"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291060070"}]},"ts":"1676291060070"} 2023-02-13 12:24:20,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,071 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,072 DEBUG [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/family1 2023-02-13 12:24:20,072 DEBUG [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/family1 2023-02-13 12:24:20,073 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-13 12:24:20,073 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 956d36d3510f1e7466a907210667b173, server=jenkins-hbase16.apache.org,38471,1676291011293 in 199 msec 2023-02-13 12:24:20,073 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 976a16fc4c9a9b309025ba2a32e7ab08 columnFamilyName family1 2023-02-13 12:24:20,074 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] regionserver.HStore(310): Store=976a16fc4c9a9b309025ba2a32e7ab08/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=956d36d3510f1e7466a907210667b173, ASSIGN in 366 msec 2023-02-13 12:24:20,074 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,075 DEBUG [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/family2 2023-02-13 12:24:20,075 DEBUG [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/family2 2023-02-13 12:24:20,076 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 976a16fc4c9a9b309025ba2a32e7ab08 columnFamilyName family2 2023-02-13 12:24:20,077 INFO [StoreOpener-976a16fc4c9a9b309025ba2a32e7ab08-1] regionserver.HStore(310): Store=976a16fc4c9a9b309025ba2a32e7ab08/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:20,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:24:20,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:24:20,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:20,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 976a16fc4c9a9b309025ba2a32e7ab08; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59498206, jitterRate=-0.11340764164924622}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:24:20,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 976a16fc4c9a9b309025ba2a32e7ab08: Running coprocessor pre-open hook at 1676291060070Writing region info on filesystem at 1676291060070Initializing all the Stores at 1676291060071 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060071Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291060071Cleaning up temporary data from old regions at 1676291060079 (+8 ms)Cleaning up detritus from prior splits at 1676291060079Running coprocessor post-open hooks at 1676291060084 (+5 ms)Region opened successfully at 1676291060084 2023-02-13 12:24:20,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08., pid=33, masterSystemTime=1676291060024 2023-02-13 12:24:20,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:20,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:20,087 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=976a16fc4c9a9b309025ba2a32e7ab08, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:20,087 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676291060087"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291060087"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291060087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291060087"}]},"ts":"1676291060087"} 2023-02-13 12:24:20,091 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-13 12:24:20,091 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 976a16fc4c9a9b309025ba2a32e7ab08, server=jenkins-hbase16.apache.org,38471,1676291011293 in 216 msec 2023-02-13 12:24:20,093 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=27 2023-02-13 12:24:20,093 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=976a16fc4c9a9b309025ba2a32e7ab08, ASSIGN in 384 msec 2023-02-13 12:24:20,094 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:24:20,094 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291060094"}]},"ts":"1676291060094"} 2023-02-13 12:24:20,095 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-13 12:24:20,132 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:24:20,134 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.3350 sec 2023-02-13 12:24:20,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-13 12:24:20,911 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-13 12:24:20,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:20,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,928 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 12:24:20,928 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:20,928 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 12:24:20,928 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 12:24:20,929 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,929 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,939 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:20,939 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:20,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:20,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:20,940 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:20,940 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:20,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:20,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:20,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 12:24:20,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:20,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 12:24:20,944 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:20,945 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. started... 2023-02-13 12:24:20,945 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:20,945 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:24:20,946 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:20,946 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:20,946 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. started... 2023-02-13 12:24:20,945 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. started... 2023-02-13 12:24:20,946 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 976a16fc4c9a9b309025ba2a32e7ab08: 2023-02-13 12:24:20,948 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 956d36d3510f1e7466a907210667b173: 2023-02-13 12:24:20,948 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:20,947 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 4c42cd6a08dfd848a6e7ca94fad32a2a 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-13 12:24:20,948 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:24:20,948 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:20,948 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:24:21,350 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-13 12:24:21,371 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213216fa3e67b77445dac2f18933e937fe7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213216fa3e67b77445dac2f18933e937fe7 2023-02-13 12:24:21,779 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp/family1/9f44c7dd46f54738924b30118eab803c 2023-02-13 12:24:21,790 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp/family1/9f44c7dd46f54738924b30118eab803c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c 2023-02-13 12:24:21,797 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 12:24:21,798 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 4c42cd6a08dfd848a6e7ca94fad32a2a in 851ms, sequenceid=5, compaction requested=false 2023-02-13 12:24:21,799 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: 2023-02-13 12:24:21,799 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:21,799 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:24:21,799 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:24:21,799 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,799 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 12:24:21,799 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 12:24:21,831 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:21,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:21,832 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,832 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:21,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:21,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:21,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 12:24:21,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@782d0fe0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:21,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 12:24:21,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,843 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,843 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 12:24:21,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 12:24:21,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:21,844 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,844 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:21,851 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,851 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 12:24:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:21,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:21,851 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:21,851 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 12:24:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:21,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 12:24:21,855 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 12:24:21,855 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:21,855 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:21,855 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,855 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:21,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:21,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:21,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:21,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:21,865 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:21,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,866 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:21,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:21,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 12:24:21,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:21,896 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-13 12:24:21,896 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,896 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:21,897 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:21,897 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:21,897 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:21,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:21,897 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:21,897 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:21,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:21,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:21,997 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:22,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:22,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:22,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:22,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,017 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-13 12:24:22,017 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:22,017 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-13 12:24:22,017 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-13 12:24:22,017 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,018 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:22,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:22,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:22,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:22,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:22,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-13 12:24:22,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,032 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-13 12:24:22,032 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:22,032 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. started... 2023-02-13 12:24:22,032 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:24:22,032 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 976a16fc4c9a9b309025ba2a32e7ab08: 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:22,033 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:24:22,032 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. started... 2023-02-13 12:24:22,033 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:24:22,033 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. started... 2023-02-13 12:24:22,033 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 956d36d3510f1e7466a907210667b173: 2023-02-13 12:24:22,033 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 4c42cd6a08dfd848a6e7ca94fad32a2a 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-13 12:24:22,033 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:24:22,033 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:24:22,454 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213eb7776f489054da1a0e3b69fc5c058f9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213eb7776f489054da1a0e3b69fc5c058f9 2023-02-13 12:24:22,865 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp/family1/2f5e38af74174594b9745bfaf30bd98e 2023-02-13 12:24:22,879 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp/family1/2f5e38af74174594b9745bfaf30bd98e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e 2023-02-13 12:24:22,887 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e, entries=2, sequenceid=9, filesize=5.0 K 2023-02-13 12:24:22,889 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 4c42cd6a08dfd848a6e7ca94fad32a2a in 856ms, sequenceid=9, compaction requested=false 2023-02-13 12:24:22,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: 2023-02-13 12:24:22,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:22,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:24:22,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:24:22,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-13 12:24:22,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-13 12:24:22,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,922 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:22,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:22,923 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:22,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:22,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:22,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-13 12:24:22,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3caec8b9[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:22,927 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-13 12:24:22,927 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,935 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-13 12:24:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-13 12:24:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:22,935 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,936 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:22,943 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-13 12:24:22,943 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:22,943 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-13 12:24:22,943 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:22,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:22,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:22,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:22,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:22,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:22,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-13 12:24:22,953 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-13 12:24:22,953 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:22,953 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:22,953 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,953 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:22,964 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,964 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:22,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,964 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:22,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:22,965 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:22,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,981 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:22,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:22,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:22,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-13 12:24:22,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:22,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:22,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:22,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:23,082 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:23,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:23,097 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-13 12:24:23,099 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 12:24:23,100 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 12:24:23,101 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:23,101 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291663101 2023-02-13 12:24:23,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:23,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-13 12:24:23,102 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 12:24:23,102 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x7f9e466d to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:23,115 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f842d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:23,115 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 12:24:23,948 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021390f9d4a64bca4a55909008ecf86422b1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021390f9d4a64bca4a55909008ecf86422b1 2023-02-13 12:24:23,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 12:24:23,963 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:23,965 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:52130, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:23,972 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:23,975 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:36364, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:23,984 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/8ad8cdd013cb4a6bbddd116fce4137d5 first=Optional[k0] last=Optional[k1] 2023-02-13 12:24:23,985 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/8ad8cdd013cb4a6bbddd116fce4137d5}] 2023-02-13 12:24:23,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/8ad8cdd013cb4a6bbddd116fce4137d5 for inclusion in 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 2023-02-13 12:24:23,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-13 12:24:23,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 12:24:23,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: 2023-02-13 12:24:23,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213/family1/8ad8cdd013cb4a6bbddd116fce4137d5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testScannerOnBulkLoadRefHFiles__fsjaloqsrp16t92durkcdi06q688u6mank0kjjokug8ip193obdva4i3oop7hqpu/family1/8ad8cdd013cb4a6bbddd116fce4137d5 2023-02-13 12:24:23,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testScannerOnBulkLoadRefHFiles__fsjaloqsrp16t92durkcdi06q688u6mank0kjjokug8ip193obdva4i3oop7hqpu/family1/8ad8cdd013cb4a6bbddd116fce4137d5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_ 2023-02-13 12:24:23,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testScannerOnBulkLoadRefHFiles__fsjaloqsrp16t92durkcdi06q688u6mank0kjjokug8ip193obdva4i3oop7hqpu/family1/8ad8cdd013cb4a6bbddd116fce4137d5 into 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_ - updating store file list. 2023-02-13 12:24:24,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_ into 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 2023-02-13 12:24:24,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testScannerOnBulkLoadRefHFiles__fsjaloqsrp16t92durkcdi06q688u6mank0kjjokug8ip193obdva4i3oop7hqpu/family1/8ad8cdd013cb4a6bbddd116fce4137d5 into 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_) 2023-02-13 12:24:24,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testScannerOnBulkLoadRefHFiles__fsjaloqsrp16t92durkcdi06q688u6mank0kjjokug8ip193obdva4i3oop7hqpu/family1/8ad8cdd013cb4a6bbddd116fce4137d5 2023-02-13 12:24:24,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:24:24,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:24:24,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 is initiating minor compaction (all files) 2023-02-13 12:24:24,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64474733; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:24,004 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 in testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:24:24,004 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp, totalSize=15.1 K 2023-02-13 12:24:24,005 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 9f44c7dd46f54738924b30118eab803c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291060912 2023-02-13 12:24:24,006 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 2f5e38af74174594b9745bfaf30bd98e, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291060912 2023-02-13 12:24:24,006 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291060912 2023-02-13 12:24:24,007 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 12:24:24,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:24,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213216fa3e67b77445dac2f18933e937fe7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213216fa3e67b77445dac2f18933e937fe7 2023-02-13 12:24:24,012 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213eb7776f489054da1a0e3b69fc5c058f9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230213eb7776f489054da1a0e3b69fc5c058f9 2023-02-13 12:24:24,012 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-13 12:24:24,012 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 12:24:24,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-13 12:24:24,013 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-13 12:24:24,013 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:24,014 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-13 12:24:24,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 916 msec 2023-02-13 12:24:24,018 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): 4c42cd6a08dfd848a6e7ca94fad32a2a#family1#compaction#38 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:24:24,053 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=245 (was 250), OpenFileDescriptor=584 (was 581) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 170), ProcessCount=170 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=6085 (was 6087) 2023-02-13 12:24:24,065 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=245, OpenFileDescriptor=584, MaxFileDescriptor=60000, SystemLoadAverage=156, ProcessCount=170, AvailableMemoryMB=6085 2023-02-13 12:24:24,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:24:24,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:24,070 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:24:24,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-13 12:24:24,071 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:24:24,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:24,073 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:24,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89 empty. 2023-02-13 12:24:24,074 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:24,074 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-13 12:24:24,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:24,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:24,487 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:24,490 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 73394efe2c0767e17a09a439425afd89, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:24,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:24,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:24,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 73394efe2c0767e17a09a439425afd89, disabling compactions & flushes 2023-02-13 12:24:24,902 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:24,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:24,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. after waiting 0 ms 2023-02-13 12:24:24,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:24,902 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:24,903 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 73394efe2c0767e17a09a439425afd89: Waiting for close lock at 1676291064902Disabling compacts and flushes for region at 1676291064902Disabling writes for close at 1676291064902Writing region close event to WAL at 1676291064902Closed at 1676291064902 2023-02-13 12:24:24,909 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:24:24,911 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291064911"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291064911"}]},"ts":"1676291064911"} 2023-02-13 12:24:24,915 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:24:24,916 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:24:24,916 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291064916"}]},"ts":"1676291064916"} 2023-02-13 12:24:24,917 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-13 12:24:24,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=73394efe2c0767e17a09a439425afd89, ASSIGN}] 2023-02-13 12:24:24,967 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=73394efe2c0767e17a09a439425afd89, ASSIGN 2023-02-13 12:24:24,968 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=73394efe2c0767e17a09a439425afd89, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:25,119 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=73394efe2c0767e17a09a439425afd89, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:25,120 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291065119"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291065119"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291065119"}]},"ts":"1676291065119"} 2023-02-13 12:24:25,122 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 73394efe2c0767e17a09a439425afd89, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:25,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:25,261 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/.tmp/family1/9080a621bf1d4d169be3d754ae17a71d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9080a621bf1d4d169be3d754ae17a71d 2023-02-13 12:24:25,268 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 4c42cd6a08dfd848a6e7ca94fad32a2a/family1 of 4c42cd6a08dfd848a6e7ca94fad32a2a into 9080a621bf1d4d169be3d754ae17a71d(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:24:25,268 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: 2023-02-13 12:24:25,268 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1676291064004; duration=1sec 2023-02-13 12:24:25,268 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:25,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:25,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 73394efe2c0767e17a09a439425afd89, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:25,277 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:25,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,279 INFO [StoreOpener-73394efe2c0767e17a09a439425afd89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,282 DEBUG [StoreOpener-73394efe2c0767e17a09a439425afd89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:25,282 DEBUG [StoreOpener-73394efe2c0767e17a09a439425afd89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:25,282 INFO [StoreOpener-73394efe2c0767e17a09a439425afd89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73394efe2c0767e17a09a439425afd89 columnFamilyName family1 2023-02-13 12:24:25,285 INFO [StoreOpener-73394efe2c0767e17a09a439425afd89-1] regionserver.HStore(310): Store=73394efe2c0767e17a09a439425afd89/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:25,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 73394efe2c0767e17a09a439425afd89 2023-02-13 12:24:25,293 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:25,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 73394efe2c0767e17a09a439425afd89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69821352, jitterRate=0.04041922092437744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:25,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 73394efe2c0767e17a09a439425afd89: Running coprocessor pre-open hook at 1676291065278Writing region info on filesystem at 1676291065278Initializing all the Stores at 1676291065279 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291065279Cleaning up temporary data from old regions at 1676291065288 (+9 ms)Cleaning up detritus from prior splits at 1676291065288Running coprocessor post-open hooks at 1676291065294 (+6 ms)Region opened successfully at 1676291065294 2023-02-13 12:24:25,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89., pid=37, masterSystemTime=1676291065274 2023-02-13 12:24:25,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:25,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:25,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=73394efe2c0767e17a09a439425afd89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:25,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676291065298"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291065298"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291065298"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291065298"}]},"ts":"1676291065298"} 2023-02-13 12:24:25,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-13 12:24:25,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 73394efe2c0767e17a09a439425afd89, server=jenkins-hbase16.apache.org,38471,1676291011293 in 178 msec 2023-02-13 12:24:25,303 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 12:24:25,303 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=73394efe2c0767e17a09a439425afd89, ASSIGN in 337 msec 2023-02-13 12:24:25,303 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:24:25,304 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291065303"}]},"ts":"1676291065303"} 2023-02-13 12:24:25,305 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-13 12:24:25,340 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:24:25,342 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.2730 sec 2023-02-13 12:24:25,371 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_] to archive 2023-02-13 12:24:25,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:25,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/9f44c7dd46f54738924b30118eab803c 2023-02-13 12:24:25,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/2f5e38af74174594b9745bfaf30bd98e 2023-02-13 12:24:25,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/family1/8823c2f3f43a400a8a0c8d78a1741bf2_SeqId_12_ 2023-02-13 12:24:25,563 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:24:26,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 12:24:26,186 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-13 12:24:26,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:26,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,217 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:26,217 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:26,217 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:26,217 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:26,218 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,218 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,230 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:26,230 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:26,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:26,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:26,231 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:26,231 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:26,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:26,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,232 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,232 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:26,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:26,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:26,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:26,233 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:26,233 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:26,233 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:26,233 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:26,233 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:24:26,662 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d3ae82d569b849bea54c8c80460c2cb8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130d3ae82d569b849bea54c8c80460c2cb8 2023-02-13 12:24:27,067 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/12d9dcbfbdc04bbabc14e9b4d283a959 2023-02-13 12:24:27,086 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/12d9dcbfbdc04bbabc14e9b4d283a959 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959 2023-02-13 12:24:27,094 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-13 12:24:27,095 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 862ms, sequenceid=5, compaction requested=false 2023-02-13 12:24:27,095 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:27,096 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:27,096 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:27,096 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:27,096 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:27,096 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,096 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:27,096 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:27,118 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,118 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:27,119 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,119 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:27,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:27,123 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74de6f43[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:27,123 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:27,123 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,131 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,131 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:27,131 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:27,131 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:27,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:27,139 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:27,139 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:27,139 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:27,139 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:27,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:27,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:27,142 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:27,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:27,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:27,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,142 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:27,151 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,151 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:27,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:27,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,151 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:27,152 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:27,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,154 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:27,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:27,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:27,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:27,164 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:27,165 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:27,165 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:27,165 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:27,165 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:27,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:27,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:27,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:27,266 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:27,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:27,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:27,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:27,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,279 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:27,279 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,289 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,289 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:27,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:27,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:27,290 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:27,290 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:27,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:27,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:27,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:27,293 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:27,293 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:27,293 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:27,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:27,294 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:27,294 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:27,294 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:27,294 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:24:27,723 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282716246cef1e4eac9be1d52942a179b2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511282716246cef1e4eac9be1d52942a179b2 2023-02-13 12:24:28,131 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/cc41449d0acb4838b36cacdff227019d 2023-02-13 12:24:28,144 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/cc41449d0acb4838b36cacdff227019d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d 2023-02-13 12:24:28,148 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:24:28,149 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 855ms, sequenceid=9, compaction requested=false 2023-02-13 12:24:28,149 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:28,149 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:28,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:28,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:28,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:28,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:28,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,181 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:28,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:28,181 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,181 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:28,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:28,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:28,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:28,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:28,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25f07337[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:28,186 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:28,186 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,219 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,220 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:28,220 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:28,220 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:28,220 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:28,236 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:28,236 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,236 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:28,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:28,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:28,236 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:28,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:28,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:28,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:28,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:28,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:28,246 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:28,246 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:28,246 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:28,246 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,246 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:28,255 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,255 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:28,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,256 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:28,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:28,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:28,257 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:28,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,259 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:28,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:28,268 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,268 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:28,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:28,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:28,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:28,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:28,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,269 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,269 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,269 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:28,269 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:28,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:28,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:28,369 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:28,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:28,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:28,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:28,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,382 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:28,382 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:28,383 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:28,383 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:28,383 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,383 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,393 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,393 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:28,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:28,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:28,393 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:28,393 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:28,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:28,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,394 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:28,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:28,394 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:28,394 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:28,394 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:28,395 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:28,395 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:28,395 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:28,395 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:28,395 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:28,817 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120135d459893a4c4e61aac538f052b1fd72 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120135d459893a4c4e61aac538f052b1fd72 2023-02-13 12:24:29,225 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/42f4d9ec748248c7ae064b4779982cb4 2023-02-13 12:24:29,238 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/42f4d9ec748248c7ae064b4779982cb4 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4 2023-02-13 12:24:29,245 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 12:24:29,246 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 851ms, sequenceid=13, compaction requested=true 2023-02-13 12:24:29,247 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:29,248 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:29,248 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:29,248 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:29,248 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,248 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:29,248 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:29,281 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,281 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:29,282 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:29,282 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:29,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:29,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:29,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:29,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35b22a66[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:29,287 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:29,287 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,297 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:29,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:29,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:29,298 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,299 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:29,310 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:29,310 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:29,310 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:29,310 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:29,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:29,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:29,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:29,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:29,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:29,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:29,320 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:29,320 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:29,320 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:29,320 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,320 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:29,331 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,331 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,331 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:29,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:29,332 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:29,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,335 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:29,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,347 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:29,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:29,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:29,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:29,348 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:29,347 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,348 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,348 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:29,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:29,348 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,448 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:29,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:29,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:29,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:29,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,455 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:29,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:29,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:29,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:29,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,464 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:29,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:29,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:29,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,465 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:29,465 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:29,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:29,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:29,465 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:29,466 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:29,466 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:29,466 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:29,466 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:29,466 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:29,886 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055ac0e3ff88624b0d845b6a6f9d1bfeab to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055ac0e3ff88624b0d845b6a6f9d1bfeab 2023-02-13 12:24:30,293 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/548c9a8878b345c4aaa0ed4964b56ecb 2023-02-13 12:24:30,298 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/548c9a8878b345c4aaa0ed4964b56ecb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb 2023-02-13 12:24:30,302 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 12:24:30,303 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 837ms, sequenceid=17, compaction requested=true 2023-02-13 12:24:30,303 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:30,303 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:30,303 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:30,303 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:30,303 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,303 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:30,304 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:30,339 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:30,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:30,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:30,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:30,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:30,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:30,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f13c769[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:30,343 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:30,343 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,351 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:30,351 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:30,352 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:30,352 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,352 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:30,359 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:30,359 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,359 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:30,360 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:30,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:30,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:30,366 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:30,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:30,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:30,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,366 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:30,376 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,376 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:30,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,376 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:30,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:30,377 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:30,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,380 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:30,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:30,389 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:30,389 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:30,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:30,389 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,390 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:30,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:30,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:30,390 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:30,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,491 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:30,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:30,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:30,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:30,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,507 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:30,507 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:30,507 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:30,507 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:30,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,508 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,540 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:30,540 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:30,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:30,541 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:30,541 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:30,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:30,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:30,544 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:30,545 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:30,545 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:30,545 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:30,545 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:30,545 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:30,545 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:30,545 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:30,546 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:30,782 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:24:30,970 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228481bfcdc73db4010a79bc9e2a6995f0d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228481bfcdc73db4010a79bc9e2a6995f0d 2023-02-13 12:24:31,378 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/5289bd23549e4fcaa71c900129c83d81 2023-02-13 12:24:31,386 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/5289bd23549e4fcaa71c900129c83d81 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81 2023-02-13 12:24:31,391 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 12:24:31,392 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 846ms, sequenceid=21, compaction requested=true 2023-02-13 12:24:31,392 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:31,392 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:31,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:31,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:31,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:31,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:31,431 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:31,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:31,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,432 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:31,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:31,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:31,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:31,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78b01bbf[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:31,436 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:31,437 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,447 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,448 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:31,448 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:31,448 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:31,448 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,448 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:31,460 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,460 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:31,460 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:31,460 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:31,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:31,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:31,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:31,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:31,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:31,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:31,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:31,471 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:31,472 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:31,472 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:31,472 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,472 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:31,484 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,484 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,484 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:31,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:31,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:31,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:31,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:31,486 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:31,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,488 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,501 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,501 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:31,501 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:31,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:31,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:31,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:31,501 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:31,501 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,501 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:31,502 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:31,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:31,502 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,502 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:31,502 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,503 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:31,503 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,603 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:31,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:31,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:31,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:31,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,621 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:31,621 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:31,621 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:31,621 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:31,622 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,622 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,631 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,631 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:31,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:31,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:31,632 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:31,632 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:31,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,633 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:31,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,635 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,635 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:31,635 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:31,635 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:31,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:31,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:31,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:31,636 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:31,636 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:31,636 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:31,637 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:31,659 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231510ff3ca97da4022b959c93b45e322f1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231510ff3ca97da4022b959c93b45e322f1 2023-02-13 12:24:32,068 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/f33bd6b846d2446b85d1791dee62d2da 2023-02-13 12:24:32,080 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/f33bd6b846d2446b85d1791dee62d2da as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da 2023-02-13 12:24:32,085 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 12:24:32,086 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 449ms, sequenceid=25, compaction requested=true 2023-02-13 12:24:32,086 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:32,086 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:32,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:32,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:32,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:32,086 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:32,122 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,122 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:32,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:32,123 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,123 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:32,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:32,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:32,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:32,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:32,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e7fb655[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:32,127 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:32,127 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,134 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,135 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:32,135 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:32,135 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:32,135 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,135 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:32,143 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,143 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:32,143 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:32,143 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:32,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:32,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:32,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:32,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:32,151 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:32,151 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:32,151 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:32,151 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,151 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:32,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:32,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:32,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,164 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:32,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:32,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:32,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:32,165 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:32,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,170 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:32,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,180 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:32,180 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:32,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:32,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:32,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:32,180 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,180 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:32,181 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:32,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,181 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:32,281 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:32,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:32,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:32,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:32,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,298 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:32,298 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:32,298 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:32,298 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:32,299 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,299 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,309 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:32,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:32,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:32,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:32,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:32,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:32,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:32,311 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:32,311 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:32,311 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:32,311 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:32,311 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:32,311 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:32,311 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:32,311 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:32,734 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018a0e9f34ff67488aa22a5dd17c1a49b8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601018a0e9f34ff67488aa22a5dd17c1a49b8 2023-02-13 12:24:33,142 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/faa3c4cc23ff494fa5fc2cb954e68cd2 2023-02-13 12:24:33,151 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/faa3c4cc23ff494fa5fc2cb954e68cd2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2 2023-02-13 12:24:33,158 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 12:24:33,159 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 848ms, sequenceid=29, compaction requested=true 2023-02-13 12:24:33,159 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:33,159 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:33,159 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:33,159 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:33,159 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,159 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:33,159 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:33,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,197 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:33,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:33,205 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,205 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:33,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:33,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:33,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:33,207 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:33,207 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a3a56e6[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:33,217 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,218 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:33,218 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:33,218 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:33,218 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,218 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:33,226 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:33,226 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,226 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:33,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:33,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:33,226 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:33,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:33,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:33,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:33,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:33,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:33,232 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:33,232 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:33,232 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:33,232 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,232 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:33,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,243 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:33,244 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:33,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,279 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,279 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:33,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:33,279 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:33,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:33,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:33,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:33,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:33,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:33,281 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:33,281 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:33,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:33,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:33,381 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:33,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:33,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:33,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:33,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,396 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:33,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:33,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:33,396 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:33,397 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,397 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,409 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:33,409 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:33,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:33,410 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:33,410 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:33,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:33,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,410 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:33,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:33,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:33,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:33,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:33,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:33,411 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:33,411 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:33,411 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:33,411 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:33,832 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035ac18bd7a7b34930b29f7562352d90a2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ac18bd7a7b34930b29f7562352d90a2 2023-02-13 12:24:34,242 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/4165adb88c8a4dec908f78d6b1b93d21 2023-02-13 12:24:34,254 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/4165adb88c8a4dec908f78d6b1b93d21 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21 2023-02-13 12:24:34,261 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 12:24:34,262 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 851ms, sequenceid=33, compaction requested=true 2023-02-13 12:24:34,262 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:34,262 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:34,262 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:34,262 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:34,262 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,262 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:34,262 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:34,297 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,297 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:34,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:34,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:34,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:34,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bf3fe47[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:34,301 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:34,301 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,309 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,309 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:34,310 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:34,310 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:34,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:34,318 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:34,318 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,318 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:34,318 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:34,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:34,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:34,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:34,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:34,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:34,326 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:34,326 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:34,326 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:34,326 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,326 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:34,334 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,334 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:34,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:34,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,335 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:34,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:34,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:34,336 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:34,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:34,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,340 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:34,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:34,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:34,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:34,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:34,351 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:34,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,351 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,451 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:34,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:34,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:34,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:34,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,463 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:34,463 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:34,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:34,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:34,465 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,465 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,476 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:34,476 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:34,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:34,477 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:34,477 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:34,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:34,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,479 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:34,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:34,479 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:34,479 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:34,479 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:34,480 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:34,480 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:34,480 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:34,480 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:34,480 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:34,908 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907c716c73e44d445f883d07afb5a38a6a4 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907c716c73e44d445f883d07afb5a38a6a4 2023-02-13 12:24:35,319 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/be59ba1286dd44379ca08ee86f4f1e65 2023-02-13 12:24:35,330 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/be59ba1286dd44379ca08ee86f4f1e65 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65 2023-02-13 12:24:35,334 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 12:24:35,335 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 855ms, sequenceid=37, compaction requested=true 2023-02-13 12:24:35,335 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:35,335 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:35,335 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:35,335 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:35,335 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,335 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:35,335 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:35,372 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:35,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:35,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:35,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:35,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:35,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:35,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:35,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63666365[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:35,375 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:35,375 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,384 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,384 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:35,385 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:35,385 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:35,385 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,385 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:35,392 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:35,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:35,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:35,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:35,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:35,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:35,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:35,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:35,398 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:35,398 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:35,398 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:35,399 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,399 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:35,409 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,409 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:35,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:35,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,410 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:35,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:35,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:35,411 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:35,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,414 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:35,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,426 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:35,426 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:35,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:35,426 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:35,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:35,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:35,427 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:35,427 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:35,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:35,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:35,528 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:35,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:35,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:35,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:35,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,543 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:35,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:35,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-13 12:24:35,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-13 12:24:35,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,543 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,551 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,551 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:35,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:35,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:35,551 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:35,551 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:35,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:35,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-13 12:24:35,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:35,552 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-13 12:24:35,552 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:35,552 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-13 12:24:35,552 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:35,552 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:35,552 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:35,552 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. started... 2023-02-13 12:24:35,553 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 73394efe2c0767e17a09a439425afd89 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:35,974 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112060627ca0846543639c12174ab9b2c22e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112060627ca0846543639c12174ab9b2c22e 2023-02-13 12:24:36,387 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/26ccf1c7b61b47c6b960b583d84c0c66 2023-02-13 12:24:36,400 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/26ccf1c7b61b47c6b960b583d84c0c66 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66 2023-02-13 12:24:36,407 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 12:24:36,408 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 73394efe2c0767e17a09a439425afd89 in 856ms, sequenceid=41, compaction requested=true 2023-02-13 12:24:36,409 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:36,409 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:36,409 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:36,409 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:36,409 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,409 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-13 12:24:36,409 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-13 12:24:36,417 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,417 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:36,418 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,418 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:36,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:36,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:36,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-13 12:24:36,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4088bc42[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:36,421 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-13 12:24:36,421 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,430 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-13 12:24:36,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-13 12:24:36,431 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,431 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:36,431 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:36,439 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,439 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-13 12:24:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:36,439 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:36,439 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-13 12:24:36,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:36,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:36,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:36,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:36,446 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:24:36,446 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-13 12:24:36,447 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:36,447 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:36,447 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,447 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:36,459 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,459 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:36,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,460 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:36,461 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:36,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:36,464 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:36,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,497 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:36,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-13 12:24:36,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,498 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:36,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:36,498 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:36,498 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:36,497 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:36,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-13 12:24:36,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:36,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:36,598 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:36,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:36,604 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-13 12:24:36,608 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:36,610 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:36,611 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:36,611 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291676611 2023-02-13 12:24:36,613 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:36,615 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-13 12:24:36,615 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:36,615 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x45ca40f3 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:36,631 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39b8aa8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:36,631 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,478 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030fbb83943994481b82a4bfcde1d4f897 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601030fbb83943994481b82a4bfcde1d4f897 2023-02-13 12:24:37,480 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:37,496 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:37,497 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44012, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:37,504 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:37,505 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:59900, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:37,514 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6fe38491f2a54c1f98ff47845fe6532a first=Optional[r3] last=Optional[r6] 2023-02-13 12:24:37,515 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6fe38491f2a54c1f98ff47845fe6532a}] 2023-02-13 12:24:37,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6fe38491f2a54c1f98ff47845fe6532a for inclusion in 73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:37,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 12:24:37,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:37,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:37,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/6fe38491f2a54c1f98ff47845fe6532a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ito40rtup1a4c0r3sh9alnqkjcn2hu72op7jgurg4ircu016slh5vc3h64ojarss/family1/6fe38491f2a54c1f98ff47845fe6532a 2023-02-13 12:24:37,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ito40rtup1a4c0r3sh9alnqkjcn2hu72op7jgurg4ircu016slh5vc3h64ojarss/family1/6fe38491f2a54c1f98ff47845fe6532a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_ 2023-02-13 12:24:37,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ito40rtup1a4c0r3sh9alnqkjcn2hu72op7jgurg4ircu016slh5vc3h64ojarss/family1/6fe38491f2a54c1f98ff47845fe6532a into 73394efe2c0767e17a09a439425afd89/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_ - updating store file list. 2023-02-13 12:24:37,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_ into 73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:37,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ito40rtup1a4c0r3sh9alnqkjcn2hu72op7jgurg4ircu016slh5vc3h64ojarss/family1/6fe38491f2a54c1f98ff47845fe6532a into 73394efe2c0767e17a09a439425afd89/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_) 2023-02-13 12:24:37,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ito40rtup1a4c0r3sh9alnqkjcn2hu72op7jgurg4ircu016slh5vc3h64ojarss/family1/6fe38491f2a54c1f98ff47845fe6532a 2023-02-13 12:24:37,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:24:37,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:24:37,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 73394efe2c0767e17a09a439425afd89/family1 is initiating minor compaction 2023-02-13 12:24:37,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7ac055f1; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:37,541 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 73394efe2c0767e17a09a439425afd89/family1 in testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:37,541 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp, totalSize=50.0 K 2023-02-13 12:24:37,541 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting cc41449d0acb4838b36cacdff227019d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:24:37,541 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 42f4d9ec748248c7ae064b4779982cb4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:24:37,541 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 548c9a8878b345c4aaa0ed4964b56ecb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 5289bd23549e4fcaa71c900129c83d81, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting f33bd6b846d2446b85d1791dee62d2da, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting faa3c4cc23ff494fa5fc2cb954e68cd2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 4165adb88c8a4dec908f78d6b1b93d21, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting be59ba1286dd44379ca08ee86f4f1e65, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 26ccf1c7b61b47c6b960b583d84c0c66, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:24:37,542 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 35a855f8b9764997adf3adfa425fab65_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:24:37,544 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:37,546 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:37,547 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228481bfcdc73db4010a79bc9e2a6995f0d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228481bfcdc73db4010a79bc9e2a6995f0d 2023-02-13 12:24:37,549 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231510ff3ca97da4022b959c93b45e322f1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231510ff3ca97da4022b959c93b45e322f1 2023-02-13 12:24:37,550 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601018a0e9f34ff67488aa22a5dd17c1a49b8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601018a0e9f34ff67488aa22a5dd17c1a49b8 2023-02-13 12:24:37,551 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ac18bd7a7b34930b29f7562352d90a2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035ac18bd7a7b34930b29f7562352d90a2 2023-02-13 12:24:37,551 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-13 12:24:37,551 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:37,572 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): 73394efe2c0767e17a09a439425afd89#family1#compaction#49 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:24:37,594 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/a0b5fd9fbe65473e86f3bcd66e124441 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/a0b5fd9fbe65473e86f3bcd66e124441 2023-02-13 12:24:37,598 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 73394efe2c0767e17a09a439425afd89/family1 of 73394efe2c0767e17a09a439425afd89 into a0b5fd9fbe65473e86f3bcd66e124441(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:37,599 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:37,599 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676291077541; duration=0sec 2023-02-13 12:24:37,599 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:37,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_] to archive 2023-02-13 12:24:37,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:37,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/cc41449d0acb4838b36cacdff227019d 2023-02-13 12:24:37,681 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/42f4d9ec748248c7ae064b4779982cb4 2023-02-13 12:24:37,682 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/548c9a8878b345c4aaa0ed4964b56ecb 2023-02-13 12:24:37,683 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/5289bd23549e4fcaa71c900129c83d81 2023-02-13 12:24:37,685 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f33bd6b846d2446b85d1791dee62d2da 2023-02-13 12:24:37,686 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/faa3c4cc23ff494fa5fc2cb954e68cd2 2023-02-13 12:24:37,688 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/4165adb88c8a4dec908f78d6b1b93d21 2023-02-13 12:24:37,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/be59ba1286dd44379ca08ee86f4f1e65 2023-02-13 12:24:37,690 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/26ccf1c7b61b47c6b960b583d84c0c66 2023-02-13 12:24:37,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/35a855f8b9764997adf3adfa425fab65_SeqId_44_ 2023-02-13 12:24:38,404 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120592b698d72d8046a49254b839687c8e3f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120592b698d72d8046a49254b839687c8e3f 2023-02-13 12:24:38,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:38,429 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28f2193e52b4a1fa47863a5d4358788 first=Optional[r00] last=Optional[r2] 2023-02-13 12:24:38,430 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28f2193e52b4a1fa47863a5d4358788}] 2023-02-13 12:24:38,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28f2193e52b4a1fa47863a5d4358788 for inclusion in 73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:38,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 12:24:38,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:38,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:38,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f28f2193e52b4a1fa47863a5d4358788 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rlut6ba6qt4t6ed40h8ql686h1n7d2a1d27sv8vsa1ai3ao9mnsd0n6074upkdvh/family1/f28f2193e52b4a1fa47863a5d4358788 2023-02-13 12:24:38,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rlut6ba6qt4t6ed40h8ql686h1n7d2a1d27sv8vsa1ai3ao9mnsd0n6074upkdvh/family1/f28f2193e52b4a1fa47863a5d4358788 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_ 2023-02-13 12:24:38,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rlut6ba6qt4t6ed40h8ql686h1n7d2a1d27sv8vsa1ai3ao9mnsd0n6074upkdvh/family1/f28f2193e52b4a1fa47863a5d4358788 into 73394efe2c0767e17a09a439425afd89/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_ - updating store file list. 2023-02-13 12:24:38,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_ into 73394efe2c0767e17a09a439425afd89/family1 2023-02-13 12:24:38,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rlut6ba6qt4t6ed40h8ql686h1n7d2a1d27sv8vsa1ai3ao9mnsd0n6074upkdvh/family1/f28f2193e52b4a1fa47863a5d4358788 into 73394efe2c0767e17a09a439425afd89/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_) 2023-02-13 12:24:38,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyPolicy__rlut6ba6qt4t6ed40h8ql686h1n7d2a1d27sv8vsa1ai3ao9mnsd0n6074upkdvh/family1/f28f2193e52b4a1fa47863a5d4358788 2023-02-13 12:24:38,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:24:38,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:24:38,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 73394efe2c0767e17a09a439425afd89/family1 is initiating minor compaction (all files) 2023-02-13 12:24:38,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@67b46060; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:38,449 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 73394efe2c0767e17a09a439425afd89/family1 in testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:24:38,449 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/a0b5fd9fbe65473e86f3bcd66e124441, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp, totalSize=337.0 K 2023-02-13 12:24:38,449 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 12d9dcbfbdc04bbabc14e9b4d283a959, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:24:38,450 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting a0b5fd9fbe65473e86f3bcd66e124441, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:24:38,450 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836144 2023-02-13 12:24:38,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:38,451 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:38,453 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130d3ae82d569b849bea54c8c80460c2cb8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130d3ae82d569b849bea54c8c80460c2cb8 2023-02-13 12:24:38,454 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120135d459893a4c4e61aac538f052b1fd72 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120135d459893a4c4e61aac538f052b1fd72 2023-02-13 12:24:38,455 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055ac0e3ff88624b0d845b6a6f9d1bfeab to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055ac0e3ff88624b0d845b6a6f9d1bfeab 2023-02-13 12:24:38,455 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-13 12:24:38,455 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 12:24:38,456 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-13 12:24:38,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-13 12:24:38,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:38,457 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:24:38,459 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8510 sec 2023-02-13 12:24:38,466 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 73394efe2c0767e17a09a439425afd89#family1#compaction#50 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:24:38,548 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=250 (was 245) - Thread LEAK? -, OpenFileDescriptor=607 (was 584) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=122 (was 156), ProcessCount=170 (was 170), AvailableMemoryMB=6062 (was 6085) 2023-02-13 12:24:38,559 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=249, OpenFileDescriptor=607, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=170, AvailableMemoryMB=6061 2023-02-13 12:24:38,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:24:38,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:38,565 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:24:38,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-13 12:24:38,565 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:24:38,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 12:24:38,567 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:38,568 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 empty. 2023-02-13 12:24:38,568 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:38,569 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-13 12:24:38,577 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:38,578 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => a3b734d5c30952e348ac86e3bc696201, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:38,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 12:24:38,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 12:24:38,984 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:38,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing a3b734d5c30952e348ac86e3bc696201, disabling compactions & flushes 2023-02-13 12:24:38,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:38,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:38,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. after waiting 0 ms 2023-02-13 12:24:38,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:38,985 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:38,985 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for a3b734d5c30952e348ac86e3bc696201: Waiting for close lock at 1676291078985Disabling compacts and flushes for region at 1676291078985Disabling writes for close at 1676291078985Writing region close event to WAL at 1676291078985Closed at 1676291078985 2023-02-13 12:24:38,989 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:24:38,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291078991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291078991"}]},"ts":"1676291078991"} 2023-02-13 12:24:38,994 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:24:38,995 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:24:38,995 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291078995"}]},"ts":"1676291078995"} 2023-02-13 12:24:38,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-13 12:24:39,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, ASSIGN}] 2023-02-13 12:24:39,044 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, ASSIGN 2023-02-13 12:24:39,045 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:39,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 12:24:39,196 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:39,197 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291079196"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291079196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291079196"}]},"ts":"1676291079196"} 2023-02-13 12:24:39,200 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:39,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:39,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3b734d5c30952e348ac86e3bc696201, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:39,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:39,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,363 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,365 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:39,365 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:39,366 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3b734d5c30952e348ac86e3bc696201 columnFamilyName family1 2023-02-13 12:24:39,367 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(310): Store=a3b734d5c30952e348ac86e3bc696201/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:39,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:39,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:39,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened a3b734d5c30952e348ac86e3bc696201; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71836640, jitterRate=0.0704493522644043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:39,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for a3b734d5c30952e348ac86e3bc696201: Running coprocessor pre-open hook at 1676291079362Writing region info on filesystem at 1676291079362Initializing all the Stores at 1676291079363 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291079363Cleaning up temporary data from old regions at 1676291079369 (+6 ms)Cleaning up detritus from prior splits at 1676291079370 (+1 ms)Running coprocessor post-open hooks at 1676291079375 (+5 ms)Region opened successfully at 1676291079375 2023-02-13 12:24:39,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., pid=41, masterSystemTime=1676291079354 2023-02-13 12:24:39,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:39,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:39,378 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:39,378 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291079378"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291079378"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291079378"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291079378"}]},"ts":"1676291079378"} 2023-02-13 12:24:39,382 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-13 12:24:39,382 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293 in 180 msec 2023-02-13 12:24:39,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-13 12:24:39,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, ASSIGN in 343 msec 2023-02-13 12:24:39,385 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:24:39,385 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291079385"}]},"ts":"1676291079385"} 2023-02-13 12:24:39,387 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-13 12:24:39,423 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:24:39,426 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 861 msec 2023-02-13 12:24:39,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-13 12:24:39,672 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-13 12:24:39,691 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/.tmp/family1/f5259478623e49b28f0e36e0d0484c81 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/f5259478623e49b28f0e36e0d0484c81 2023-02-13 12:24:39,697 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 73394efe2c0767e17a09a439425afd89/family1 of 73394efe2c0767e17a09a439425afd89 into f5259478623e49b28f0e36e0d0484c81(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:24:39,697 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 73394efe2c0767e17a09a439425afd89: 2023-02-13 12:24:39,697 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1676291078448; duration=1sec 2023-02-13 12:24:39,697 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:39,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:39,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,705 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:39,705 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:39,705 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:39,705 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:39,706 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,706 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,713 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:39,713 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:39,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:39,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:39,714 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:39,714 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:39,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:39,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,715 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,715 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:39,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:39,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:39,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:39,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:39,715 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:39,715 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:39,715 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:39,716 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:24:39,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/a0b5fd9fbe65473e86f3bcd66e124441, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_] to archive 2023-02-13 12:24:39,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:39,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/12d9dcbfbdc04bbabc14e9b4d283a959 2023-02-13 12:24:39,784 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/a0b5fd9fbe65473e86f3bcd66e124441 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/a0b5fd9fbe65473e86f3bcd66e124441 2023-02-13 12:24:39,785 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/family1/c2299c6d9eb249cc9a8a032eb8b28f1a_SeqId_47_ 2023-02-13 12:24:40,139 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306e5a6c1b8c1d4b58a433cfc614bca6e9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306e5a6c1b8c1d4b58a433cfc614bca6e9 2023-02-13 12:24:40,545 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/5126e724251144fcb790caeb8649c923 2023-02-13 12:24:40,552 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/5126e724251144fcb790caeb8649c923 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923 2023-02-13 12:24:40,558 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-13 12:24:40,559 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 843ms, sequenceid=5, compaction requested=false 2023-02-13 12:24:40,559 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:40,559 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:40,559 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:40,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:40,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:40,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:40,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:40,597 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,597 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:40,598 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,598 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:40,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:40,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:40,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:40,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:40,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68ad5fd3[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:40,600 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:40,600 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,609 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,609 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:40,609 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:40,609 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:40,609 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,609 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:40,617 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:40,617 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:40,617 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:40,617 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:40,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:40,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:40,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:40,620 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:40,620 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:40,620 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:40,620 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,620 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:40,630 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,630 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:40,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:40,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,630 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:40,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:40,631 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:40,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:40,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:40,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,632 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:40,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,642 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,642 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:40,642 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:40,642 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:40,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:40,643 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 12:24:40,642 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:40,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:40,643 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:40,643 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:40,643 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:40,644 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,644 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,644 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:40,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:40,744 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:40,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:40,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:40,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:40,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,757 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:40,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:40,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:40,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:40,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,767 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,767 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:40,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:40,767 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:40,768 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:40,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:40,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:40,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:40,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:40,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:40,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:40,769 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:40,769 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:40,769 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:24:41,194 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289d4f3e42c0b74a5388f3f7b4d77bb5d0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289d4f3e42c0b74a5388f3f7b4d77bb5d0 2023-02-13 12:24:41,214 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:41,215 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45ca40f3 to 127.0.0.1:64508 2023-02-13 12:24:41,215 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:41,215 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:41,215 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f9e466d to 127.0.0.1:64508 2023-02-13 12:24:41,215 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:41,216 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:24:41,216 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7844ff08 to 127.0.0.1:64508 2023-02-13 12:24:41,216 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:24:41,620 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/49a80ae1918f4f049a9d646df81f7347 2023-02-13 12:24:41,634 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/49a80ae1918f4f049a9d646df81f7347 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347 2023-02-13 12:24:41,640 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:24:41,641 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 872ms, sequenceid=9, compaction requested=false 2023-02-13 12:24:41,642 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:41,642 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:41,642 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:41,642 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:41,642 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,642 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:41,642 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:41,655 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:41,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:41,655 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,655 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:41,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:41,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:41,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:41,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:41,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ca88547[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:41,658 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:41,658 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,667 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,667 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:41,667 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:41,667 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:41,667 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,668 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:41,675 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:41,675 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,676 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:41,676 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:41,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:41,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:41,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:41,680 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:41,680 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:41,680 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:41,680 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,680 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:41,688 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,688 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:41,688 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:41,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:41,689 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:41,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,691 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:41,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,718 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,718 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,718 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,718 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:41,718 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:41,719 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:41,719 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:41,719 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:41,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:41,719 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:41,719 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,719 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:41,720 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:41,820 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:41,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:41,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:41,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:41,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,829 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:41,829 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:41,829 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:41,829 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:41,830 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,830 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,842 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:41,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:41,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:41,843 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:41,843 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:41,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,844 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,844 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:41,844 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:41,844 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:41,844 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:41,844 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:41,845 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:42,268 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015eca21f0f3684d32a43e830c1e1977d8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015eca21f0f3684d32a43e830c1e1977d8 2023-02-13 12:24:42,679 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/abb6e885afd04ce2aa05e0745de80ee6 2023-02-13 12:24:42,689 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/abb6e885afd04ce2aa05e0745de80ee6 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6 2023-02-13 12:24:42,695 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 12:24:42,696 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 851ms, sequenceid=13, compaction requested=true 2023-02-13 12:24:42,696 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:42,696 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:42,696 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:42,697 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:42,697 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,697 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:42,697 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:42,730 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,730 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:42,730 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,731 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:42,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:42,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:42,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:42,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:42,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c85aefc[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:42,733 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:42,733 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,742 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,742 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:42,742 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:42,742 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:42,743 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,743 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:42,751 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,751 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:42,751 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:42,751 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:42,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:42,757 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:42,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:42,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:42,757 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,757 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:42,767 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,767 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:42,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,768 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:42,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:42,769 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,773 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,784 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:42,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:42,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:42,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:42,784 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:42,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:42,785 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,785 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:42,785 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:42,785 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:42,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:42,885 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:42,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:42,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:42,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:42,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,899 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:42,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:42,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:42,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:42,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,909 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:42,909 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:42,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:42,909 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:42,909 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:42,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:42,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:42,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:42,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:42,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:42,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:42,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:42,911 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:42,911 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:42,911 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:42,911 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:43,333 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050341d395cb964c9eb1977f607286ecd9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512050341d395cb964c9eb1977f607286ecd9 2023-02-13 12:24:43,740 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/c6359e86fea54179b5928deb3e95f83e 2023-02-13 12:24:43,751 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/c6359e86fea54179b5928deb3e95f83e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e 2023-02-13 12:24:43,759 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 12:24:43,760 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 849ms, sequenceid=17, compaction requested=true 2023-02-13 12:24:43,760 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:43,760 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:43,760 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:43,797 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,797 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:43,797 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,797 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:43,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:43,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39ea7f82[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:43,799 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:43,799 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,809 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,809 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:43,809 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:43,809 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:43,809 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,809 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:43,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:43,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:43,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:43,817 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:43,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:43,821 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:43,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:43,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:43,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,821 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:43,830 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,830 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:43,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,830 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:43,831 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,833 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:43,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:43,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,842 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:43,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:43,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:43,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:43,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:43,842 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,843 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,843 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,843 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,843 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:43,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:43,843 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:43,844 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:43,944 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:43,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:43,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:43,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:43,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,959 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:43,959 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:43,959 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:43,959 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:43,960 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,960 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,986 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,986 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:43,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:43,987 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:43,987 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,988 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,990 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:43,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:43,990 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:43,991 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:43,991 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:43,991 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:43,991 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:43,991 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:43,991 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:43,991 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:44,413 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c23250d4fff44e96978f073f1326b09e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c23250d4fff44e96978f073f1326b09e 2023-02-13 12:24:44,819 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/f3f841871b124387afde6a9cd579ffe1 2023-02-13 12:24:44,830 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/f3f841871b124387afde6a9cd579ffe1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1 2023-02-13 12:24:44,834 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 12:24:44,835 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 844ms, sequenceid=21, compaction requested=true 2023-02-13 12:24:44,835 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:44,835 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:44,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:44,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:44,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:44,836 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:44,871 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,872 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:44,872 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,872 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:44,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:44,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b3bfcab[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:44,875 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:44,875 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,884 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,884 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:44,884 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:44,884 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:44,884 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,884 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:44,892 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:44,892 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:44,892 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:44,892 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:44,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:44,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:44,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,895 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:24:44,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:44,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:44,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:44,899 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:44,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:44,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:44,899 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,899 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:44,909 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,909 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:44,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,909 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:44,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:44,909 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:44,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:44,922 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:44,922 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,022 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:45,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:45,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:45,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:45,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,031 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:45,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:45,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:45,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:45,032 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,032 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,042 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,042 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:45,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:45,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:45,042 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,042 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,043 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:45,043 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,043 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:45,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:45,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:45,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:45,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:45,044 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:45,044 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:45,044 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:45,466 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231906908ba45cb48109f6aceed47f24897 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231906908ba45cb48109f6aceed47f24897 2023-02-13 12:24:45,876 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/ebec6df7a0a443edbae8e72aacef3ed9 2023-02-13 12:24:45,888 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/ebec6df7a0a443edbae8e72aacef3ed9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9 2023-02-13 12:24:45,896 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 12:24:45,897 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 853ms, sequenceid=25, compaction requested=true 2023-02-13 12:24:45,897 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:45,897 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:45,898 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:45,898 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:45,898 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,898 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:45,898 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:45,930 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,930 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:45,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:45,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:45,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:45,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:45,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1311c436[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:45,935 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:45,936 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,947 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,947 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:45,948 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:45,948 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:45,948 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,948 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:45,959 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:45,959 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:45,959 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:45,959 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:45,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:45,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:45,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:45,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:45,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:45,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:45,968 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:45,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:45,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:45,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,969 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:45,998 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,998 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:45,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,998 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:45,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:45,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:46,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:46,001 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:46,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,005 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:46,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,022 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:46,022 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:46,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:46,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:46,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,023 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,023 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:46,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:46,023 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:46,023 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:46,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:46,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:46,123 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:46,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:46,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:46,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:46,138 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,139 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:46,139 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:46,139 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:46,139 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:46,139 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,139 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,146 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,147 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:46,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:46,147 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,147 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:46,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:46,148 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:46,148 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:46,148 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:46,149 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:46,149 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:46,149 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:46,149 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:46,149 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:46,566 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101a917991798ec4d73ac5cdd778df4a1c9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a917991798ec4d73ac5cdd778df4a1c9 2023-02-13 12:24:46,972 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/740a54589b954aada81ed4098f42d7ea 2023-02-13 12:24:46,981 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/740a54589b954aada81ed4098f42d7ea as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea 2023-02-13 12:24:46,987 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 12:24:46,988 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 839ms, sequenceid=29, compaction requested=true 2023-02-13 12:24:46,988 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:46,988 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:46,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:46,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:46,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:46,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:46,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:47,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,022 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:47,023 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,023 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:47,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:47,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2726ae6[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:47,028 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:47,028 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,039 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:47,039 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:47,039 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:47,039 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,040 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:47,070 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,070 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:47,070 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:47,070 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:47,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:47,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:47,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:47,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:47,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:47,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:47,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:47,076 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:47,076 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:47,076 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:47,076 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,076 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:47,092 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,092 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,092 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:47,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:47,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:47,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:47,094 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,097 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:47,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,109 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:47,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:47,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-13 12:24:47,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:47,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:47,109 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,110 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,110 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:47,110 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,110 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:47,111 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:47,211 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:47,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:47,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:47,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:47,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,228 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:47,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:47,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:47,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:47,230 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,230 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,238 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,238 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:47,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:47,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:47,239 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:47,239 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:47,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:47,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:47,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:47,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:47,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:47,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:47,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:47,241 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:47,241 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:47,242 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:47,242 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:47,667 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032a87916be8694a7387336784959824fe to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601032a87916be8694a7387336784959824fe 2023-02-13 12:24:48,074 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/c1148469a6a143e381e651bebf8e38bc 2023-02-13 12:24:48,088 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/c1148469a6a143e381e651bebf8e38bc as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc 2023-02-13 12:24:48,092 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 12:24:48,093 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 851ms, sequenceid=33, compaction requested=true 2023-02-13 12:24:48,093 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:48,093 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:48,093 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:48,093 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:48,093 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,094 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:48,094 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:48,130 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,130 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:48,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:48,130 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,131 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:48,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:48,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:48,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@688e1345[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:48,133 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:48,133 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,142 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,142 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:48,142 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:48,142 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:48,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:48,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:48,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:48,150 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:48,150 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:48,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:48,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:48,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:48,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:48,156 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:48,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:48,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:48,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,156 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:48,167 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,167 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:48,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:48,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,167 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:48,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:48,168 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,171 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,184 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:48,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:48,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:48,184 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,185 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:48,185 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,185 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,185 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,185 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:48,285 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:48,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:48,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:48,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:48,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,300 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:48,300 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:48,300 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:48,300 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:48,301 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,301 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,309 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:48,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:48,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:48,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:48,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:48,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:48,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:48,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:48,312 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:48,312 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:48,312 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:48,312 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:48,312 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:48,312 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:48,312 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:48,312 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:48,737 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907aa8a0cc0f84340f9869b9c1d60951598 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907aa8a0cc0f84340f9869b9c1d60951598 2023-02-13 12:24:49,144 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/42f7b7b36c7b461b9bbaeb0b00413556 2023-02-13 12:24:49,157 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/42f7b7b36c7b461b9bbaeb0b00413556 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556 2023-02-13 12:24:49,164 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 12:24:49,165 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 853ms, sequenceid=37, compaction requested=true 2023-02-13 12:24:49,165 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:49,165 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:49,165 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:49,165 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:49,165 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,165 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:49,165 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:49,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,197 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:49,197 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,197 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:49,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:49,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:49,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:49,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:49,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cb4745b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:49,200 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:49,200 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,208 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,209 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:49,209 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:49,209 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:49,209 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,209 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:49,217 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,217 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:49,217 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:49,217 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:49,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:49,221 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:49,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:49,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:49,221 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,221 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:49,230 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,230 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:49,230 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:49,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:49,231 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:49,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,233 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:49,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,262 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:49,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:49,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:49,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:49,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:49,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:49,262 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:49,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:49,263 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,363 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:49,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:49,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:49,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:49,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,378 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:49,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:49,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-13 12:24:49,378 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-13 12:24:49,379 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,379 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,392 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:49,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:49,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:49,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:49,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,393 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-13 12:24:49,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:49,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-13 12:24:49,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:49,393 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-13 12:24:49,394 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:49,394 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:49,394 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:49,394 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. started... 2023-02-13 12:24:49,394 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a3b734d5c30952e348ac86e3bc696201 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:24:49,810 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120678d5c637657426f80f0700fb35b483d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678d5c637657426f80f0700fb35b483d 2023-02-13 12:24:50,219 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/34bac8f2d51542e083cfba03cb188793 2023-02-13 12:24:50,230 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/34bac8f2d51542e083cfba03cb188793 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793 2023-02-13 12:24:50,237 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 12:24:50,238 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a3b734d5c30952e348ac86e3bc696201 in 844ms, sequenceid=41, compaction requested=true 2023-02-13 12:24:50,238 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:50,238 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:50,238 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:50,238 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:50,238 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,238 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-13 12:24:50,238 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-13 12:24:50,272 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,272 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:50,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:50,273 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,273 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:50,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:50,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:50,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:50,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-13 12:24:50,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1199318f[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:50,278 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-13 12:24:50,278 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,288 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,289 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-13 12:24:50,289 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-13 12:24:50,289 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:50,289 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,289 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:50,300 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-13 12:24:50,301 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:50,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-13 12:24:50,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:50,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:50,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:50,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:50,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:50,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:50,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-13 12:24:50,311 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-13 12:24:50,311 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:50,311 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:50,311 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,311 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:50,322 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,322 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,322 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:50,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:50,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:50,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:50,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:50,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:50,323 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,325 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:50,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,338 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:50,338 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:50,338 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:50,338 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:50,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:50,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-13 12:24:50,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:50,338 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:50,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:50,339 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:50,340 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,440 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:50,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:50,445 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:50,449 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:50,451 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:50,453 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:50,453 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291690453 2023-02-13 12:24:50,456 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:50,457 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 12:24:50,457 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:50,457 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x501ae067 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:50,467 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ac68413, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:50,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,320 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010356202e2812874c4fb9f789e547f5b284 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2016010356202e2812874c4fb9f789e547f5b284 2023-02-13 12:24:51,321 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:51,337 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:51,338 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:33028, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:51,344 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:51,346 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45594, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:51,355 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/41c1919e450449e3b65dfb89674d1a5a first=Optional[r3] last=Optional[r6] 2023-02-13 12:24:51,356 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/41c1919e450449e3b65dfb89674d1a5a}] 2023-02-13 12:24:51,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/41c1919e450449e3b65dfb89674d1a5a for inclusion in a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:51,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 12:24:51,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:51,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:51,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/41c1919e450449e3b65dfb89674d1a5a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1m354i94srq1vgh576pqmkercdsp5k1hh8n9m4p6m868sqe1837r4hmfp5bc9bf2/family1/41c1919e450449e3b65dfb89674d1a5a 2023-02-13 12:24:51,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1m354i94srq1vgh576pqmkercdsp5k1hh8n9m4p6m868sqe1837r4hmfp5bc9bf2/family1/41c1919e450449e3b65dfb89674d1a5a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_ 2023-02-13 12:24:51,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1m354i94srq1vgh576pqmkercdsp5k1hh8n9m4p6m868sqe1837r4hmfp5bc9bf2/family1/41c1919e450449e3b65dfb89674d1a5a into a3b734d5c30952e348ac86e3bc696201/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_ - updating store file list. 2023-02-13 12:24:51,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_ into a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:51,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1m354i94srq1vgh576pqmkercdsp5k1hh8n9m4p6m868sqe1837r4hmfp5bc9bf2/family1/41c1919e450449e3b65dfb89674d1a5a into a3b734d5c30952e348ac86e3bc696201/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_) 2023-02-13 12:24:51,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__1m354i94srq1vgh576pqmkercdsp5k1hh8n9m4p6m868sqe1837r4hmfp5bc9bf2/family1/41c1919e450449e3b65dfb89674d1a5a 2023-02-13 12:24:51,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:24:51,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:24:51,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): a3b734d5c30952e348ac86e3bc696201/family1 is initiating minor compaction 2023-02-13 12:24:51,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@62e2597d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:51,377 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a3b734d5c30952e348ac86e3bc696201/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:51,377 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp, totalSize=50.4 K 2023-02-13 12:24:51,377 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 49a80ae1918f4f049a9d646df81f7347, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:24:51,377 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting abb6e885afd04ce2aa05e0745de80ee6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:24:51,377 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting c6359e86fea54179b5928deb3e95f83e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:24:51,377 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting f3f841871b124387afde6a9cd579ffe1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:24:51,377 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting ebec6df7a0a443edbae8e72aacef3ed9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:24:51,378 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 740a54589b954aada81ed4098f42d7ea, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:24:51,378 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting c1148469a6a143e381e651bebf8e38bc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:24:51,378 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 42f7b7b36c7b461b9bbaeb0b00413556, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:24:51,378 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 34bac8f2d51542e083cfba03cb188793, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:24:51,378 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting f71b1784382d4c3aa5860ee6a042899d_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:24:51,378 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:51,379 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:51,380 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c23250d4fff44e96978f073f1326b09e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c23250d4fff44e96978f073f1326b09e 2023-02-13 12:24:51,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231906908ba45cb48109f6aceed47f24897 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231906908ba45cb48109f6aceed47f24897 2023-02-13 12:24:51,382 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a917991798ec4d73ac5cdd778df4a1c9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a917991798ec4d73ac5cdd778df4a1c9 2023-02-13 12:24:51,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601032a87916be8694a7387336784959824fe to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601032a87916be8694a7387336784959824fe 2023-02-13 12:24:51,383 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-13 12:24:51,384 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,384 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:51,402 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): a3b734d5c30952e348ac86e3bc696201#family1#compaction#61 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:24:52,232 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/a640cb91077c409bb1d602b6fd4fd598 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/a640cb91077c409bb1d602b6fd4fd598 2023-02-13 12:24:52,235 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f4d63d788ff64b13ad9d349818510047 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4d63d788ff64b13ad9d349818510047 2023-02-13 12:24:52,236 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:52,239 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a3b734d5c30952e348ac86e3bc696201/family1 of a3b734d5c30952e348ac86e3bc696201 into a640cb91077c409bb1d602b6fd4fd598(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:52,239 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:52,239 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676291091377; duration=0sec 2023-02-13 12:24:52,240 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:52,261 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f461db3cff964590842fcbc45624367a first=Optional[r00] last=Optional[r2] 2023-02-13 12:24:52,262 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f461db3cff964590842fcbc45624367a}] 2023-02-13 12:24:52,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f461db3cff964590842fcbc45624367a for inclusion in a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:52,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 12:24:52,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:52,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:52,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f461db3cff964590842fcbc45624367a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__af6kqc5lfslhdvnrm9kel77ikokj48ctrk2c4mlhgi5ijo3ts9reqm8vib25qbk3/family1/f461db3cff964590842fcbc45624367a 2023-02-13 12:24:52,278 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_] to archive 2023-02-13 12:24:52,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__af6kqc5lfslhdvnrm9kel77ikokj48ctrk2c4mlhgi5ijo3ts9reqm8vib25qbk3/family1/f461db3cff964590842fcbc45624367a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_ 2023-02-13 12:24:52,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:52,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__af6kqc5lfslhdvnrm9kel77ikokj48ctrk2c4mlhgi5ijo3ts9reqm8vib25qbk3/family1/f461db3cff964590842fcbc45624367a into a3b734d5c30952e348ac86e3bc696201/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_ - updating store file list. 2023-02-13 12:24:52,281 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/49a80ae1918f4f049a9d646df81f7347 2023-02-13 12:24:52,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/abb6e885afd04ce2aa05e0745de80ee6 2023-02-13 12:24:52,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c6359e86fea54179b5928deb3e95f83e 2023-02-13 12:24:52,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_ into a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:52,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__af6kqc5lfslhdvnrm9kel77ikokj48ctrk2c4mlhgi5ijo3ts9reqm8vib25qbk3/family1/f461db3cff964590842fcbc45624367a into a3b734d5c30952e348ac86e3bc696201/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_) 2023-02-13 12:24:52,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__af6kqc5lfslhdvnrm9kel77ikokj48ctrk2c4mlhgi5ijo3ts9reqm8vib25qbk3/family1/f461db3cff964590842fcbc45624367a 2023-02-13 12:24:52,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:24:52,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f3f841871b124387afde6a9cd579ffe1 2023-02-13 12:24:52,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:24:52,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): a3b734d5c30952e348ac86e3bc696201/family1 is initiating minor compaction (all files) 2023-02-13 12:24:52,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5886316b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:52,285 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a3b734d5c30952e348ac86e3bc696201/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:52,285 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/a640cb91077c409bb1d602b6fd4fd598, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp, totalSize=390.1 K 2023-02-13 12:24:52,285 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 5126e724251144fcb790caeb8649c923, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:24:52,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/ebec6df7a0a443edbae8e72aacef3ed9 2023-02-13 12:24:52,286 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting a640cb91077c409bb1d602b6fd4fd598, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:24:52,286 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting f071a543b18a4c8aa3e584652979721f_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836144 2023-02-13 12:24:52,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/740a54589b954aada81ed4098f42d7ea 2023-02-13 12:24:52,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c1148469a6a143e381e651bebf8e38bc 2023-02-13 12:24:52,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:52,289 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:52,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/42f7b7b36c7b461b9bbaeb0b00413556 2023-02-13 12:24:52,290 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306e5a6c1b8c1d4b58a433cfc614bca6e9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511306e5a6c1b8c1d4b58a433cfc614bca6e9 2023-02-13 12:24:52,290 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/34bac8f2d51542e083cfba03cb188793 2023-02-13 12:24:52,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f71b1784382d4c3aa5860ee6a042899d_SeqId_44_ 2023-02-13 12:24:52,292 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015eca21f0f3684d32a43e830c1e1977d8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512015eca21f0f3684d32a43e830c1e1977d8 2023-02-13 12:24:52,293 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512050341d395cb964c9eb1977f607286ecd9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512050341d395cb964c9eb1977f607286ecd9 2023-02-13 12:24:52,293 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-13 12:24:52,293 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 12:24:52,294 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:52,294 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:52,294 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:52,295 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:52,296 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8480 sec 2023-02-13 12:24:52,301 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): a3b734d5c30952e348ac86e3bc696201#family1#compaction#62 average throughput is 30.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:24:52,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 12:24:52,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:52,790 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 12:24:52,793 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:52,794 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 12:24:52,799 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 12:24:52,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE}] 2023-02-13 12:24:52,804 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE 2023-02-13 12:24:52,804 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:52,805 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291092804"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291092804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291092804"}]},"ts":"1676291092804"} 2023-02-13 12:24:52,806 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:52,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:52,959 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing a3b734d5c30952e348ac86e3bc696201, disabling compactions & flushes 2023-02-13 12:24:52,959 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,129 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/.tmp/family1/fea59be67cef4de5b30cc94de6e3e91a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/fea59be67cef4de5b30cc94de6e3e91a 2023-02-13 12:24:53,136 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a3b734d5c30952e348ac86e3bc696201/family1 of a3b734d5c30952e348ac86e3bc696201 into fea59be67cef4de5b30cc94de6e3e91a(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:24:53,137 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:53,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. after waiting 0 ms 2023-02-13 12:24:53,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,137 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1676291092285; duration=0sec 2023-02-13 12:24:53,137 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:24:53,139 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/a640cb91077c409bb1d602b6fd4fd598, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_] to archive 2023-02-13 12:24:53,140 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:53,143 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/5126e724251144fcb790caeb8649c923 2023-02-13 12:24:53,145 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/a640cb91077c409bb1d602b6fd4fd598 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/a640cb91077c409bb1d602b6fd4fd598 2023-02-13 12:24:53,148 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/f071a543b18a4c8aa3e584652979721f_SeqId_47_ 2023-02-13 12:24:53,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 12:24:53,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for a3b734d5c30952e348ac86e3bc696201: Waiting for close lock at 1676291092959Running coprocessor pre-close hooks at 1676291092959Disabling compacts and flushes for region at 1676291092959Disabling writes for close at 1676291093137 (+178 ms)Writing region close event to WAL at 1676291093152 (+15 ms)Running coprocessor post-close hooks at 1676291093157 (+5 ms)Closed at 1676291093158 (+1 ms) 2023-02-13 12:24:53,158 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a3b734d5c30952e348ac86e3bc696201 to self. 2023-02-13 12:24:53,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,160 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=CLOSED 2023-02-13 12:24:53,161 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291093160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291093160"}]},"ts":"1676291093160"} 2023-02-13 12:24:53,164 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 12:24:53,164 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293 in 356 msec 2023-02-13 12:24:53,165 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=true 2023-02-13 12:24:53,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:53,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291093316"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291093316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291093316"}]},"ts":"1676291093316"} 2023-02-13 12:24:53,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:53,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3b734d5c30952e348ac86e3bc696201, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:53,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:53,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,485 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,487 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:53,487 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:53,487 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3b734d5c30952e348ac86e3bc696201 columnFamilyName family1 2023-02-13 12:24:53,495 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/fea59be67cef4de5b30cc94de6e3e91a 2023-02-13 12:24:53,495 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(310): Store=a3b734d5c30952e348ac86e3bc696201/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:53,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:53,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened a3b734d5c30952e348ac86e3bc696201; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64394788, jitterRate=-0.04044288396835327}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:53,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for a3b734d5c30952e348ac86e3bc696201: Running coprocessor pre-open hook at 1676291093483Writing region info on filesystem at 1676291093483Initializing all the Stores at 1676291093485 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676291093485Cleaning up temporary data from old regions at 1676291093499 (+14 ms)Cleaning up detritus from prior splits at 1676291093500 (+1 ms)Running coprocessor post-open hooks at 1676291093503 (+3 ms)Region opened successfully at 1676291093503 2023-02-13 12:24:53,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., pid=47, masterSystemTime=1676291093474 2023-02-13 12:24:53,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:53,506 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:53,507 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291093506"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291093506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291093506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291093506"}]},"ts":"1676291093506"} 2023-02-13 12:24:53,510 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-13 12:24:53,510 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293 in 188 msec 2023-02-13 12:24:53,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 12:24:53,512 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE in 707 msec 2023-02-13 12:24:53,514 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-13 12:24:53,514 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 714 msec 2023-02-13 12:24:53,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1370 sec 2023-02-13 12:24:53,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-13 12:24:53,517 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(278): alter status finished 2023-02-13 12:24:53,518 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:53,519 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:53,520 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:53,520 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:53,520 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291693520 2023-02-13 12:24:53,521 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:53,522 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 12:24:53,522 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:53,522 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x3a57175a to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:53,556 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@74ffea1f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:53,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:53,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:53,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:53,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:53,556 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:54,406 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a8e8c96644554a23a126cd0b50637725 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128a8e8c96644554a23a126cd0b50637725 2023-02-13 12:24:54,408 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:54,422 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:24:54,423 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:33034, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:24:54,430 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:24:54,433 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45602, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:24:54,440 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3968e4a186e24715aea85eeec91b9fff first=Optional[r05] last=Optional[r8] 2023-02-13 12:24:54,441 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=51 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3968e4a186e24715aea85eeec91b9fff}] 2023-02-13 12:24:54,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3968e4a186e24715aea85eeec91b9fff for inclusion in a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:54,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 12:24:54,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:24:54,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for a3b734d5c30952e348ac86e3bc696201: 2023-02-13 12:24:54,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3968e4a186e24715aea85eeec91b9fff to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__62kr38u1h4068fg65pelc4sb2o773hbifgen5l8sdfd22jkuaoeii5lpv32hglg8/family1/3968e4a186e24715aea85eeec91b9fff 2023-02-13 12:24:54,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__62kr38u1h4068fg65pelc4sb2o773hbifgen5l8sdfd22jkuaoeii5lpv32hglg8/family1/3968e4a186e24715aea85eeec91b9fff as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c824b259aaa94e04b6531e5bbd6bca23_SeqId_53_ 2023-02-13 12:24:54,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__62kr38u1h4068fg65pelc4sb2o773hbifgen5l8sdfd22jkuaoeii5lpv32hglg8/family1/3968e4a186e24715aea85eeec91b9fff into a3b734d5c30952e348ac86e3bc696201/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c824b259aaa94e04b6531e5bbd6bca23_SeqId_53_ - updating store file list. 2023-02-13 12:24:54,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c824b259aaa94e04b6531e5bbd6bca23_SeqId_53_ into a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:54,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__62kr38u1h4068fg65pelc4sb2o773hbifgen5l8sdfd22jkuaoeii5lpv32hglg8/family1/3968e4a186e24715aea85eeec91b9fff into a3b734d5c30952e348ac86e3bc696201/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c824b259aaa94e04b6531e5bbd6bca23_SeqId_53_) 2023-02-13 12:24:54,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__62kr38u1h4068fg65pelc4sb2o773hbifgen5l8sdfd22jkuaoeii5lpv32hglg8/family1/3968e4a186e24715aea85eeec91b9fff 2023-02-13 12:24:54,461 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:54,462 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:24:54,464 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678d5c637657426f80f0700fb35b483d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120678d5c637657426f80f0700fb35b483d 2023-02-13 12:24:54,465 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289d4f3e42c0b74a5388f3f7b4d77bb5d0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289d4f3e42c0b74a5388f3f7b4d77bb5d0 2023-02-13 12:24:54,465 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-13 12:24:54,465 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 12:24:54,465 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:54,465 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:54,466 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:54,466 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:54,466 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:54,467 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:54,468 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 949 msec 2023-02-13 12:24:54,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:24:54,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:54,934 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 12:24:54,937 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-13 12:24:54,937 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-13 12:24:54,942 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-13 12:24:54,947 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE}] 2023-02-13 12:24:54,948 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE 2023-02-13 12:24:54,949 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:54,949 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291094949"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291094949"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291094949"}]},"ts":"1676291094949"} 2023-02-13 12:24:54,951 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:55,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing a3b734d5c30952e348ac86e3bc696201, disabling compactions & flushes 2023-02-13 12:24:55,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. after waiting 0 ms 2023-02-13 12:24:55,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 12:24:55,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for a3b734d5c30952e348ac86e3bc696201: Waiting for close lock at 1676291095105Running coprocessor pre-close hooks at 1676291095105Disabling compacts and flushes for region at 1676291095105Disabling writes for close at 1676291095105Writing region close event to WAL at 1676291095112 (+7 ms)Running coprocessor post-close hooks at 1676291095120 (+8 ms)Closed at 1676291095120 2023-02-13 12:24:55,120 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a3b734d5c30952e348ac86e3bc696201 to self. 2023-02-13 12:24:55,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,122 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=CLOSED 2023-02-13 12:24:55,122 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291095122"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291095122"}]},"ts":"1676291095122"} 2023-02-13 12:24:55,124 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-13 12:24:55,124 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293 in 172 msec 2023-02-13 12:24:55,124 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=true 2023-02-13 12:24:55,275 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:55,275 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291095275"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291095275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291095275"}]},"ts":"1676291095275"} 2023-02-13 12:24:55,278 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:55,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a3b734d5c30952e348ac86e3bc696201, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:55,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:55,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,441 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,443 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:55,443 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1 2023-02-13 12:24:55,443 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a3b734d5c30952e348ac86e3bc696201 columnFamilyName family1 2023-02-13 12:24:55,448 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/c824b259aaa94e04b6531e5bbd6bca23_SeqId_53_ 2023-02-13 12:24:55,451 DEBUG [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/family1/fea59be67cef4de5b30cc94de6e3e91a 2023-02-13 12:24:55,451 INFO [StoreOpener-a3b734d5c30952e348ac86e3bc696201-1] regionserver.HStore(310): Store=a3b734d5c30952e348ac86e3bc696201/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:55,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:24:55,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened a3b734d5c30952e348ac86e3bc696201; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67357789, jitterRate=0.003709271550178528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:55,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for a3b734d5c30952e348ac86e3bc696201: Running coprocessor pre-open hook at 1676291095439Writing region info on filesystem at 1676291095439Initializing all the Stores at 1676291095441 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291095441Cleaning up temporary data from old regions at 1676291095454 (+13 ms)Cleaning up detritus from prior splits at 1676291095454Running coprocessor post-open hooks at 1676291095457 (+3 ms)Region opened successfully at 1676291095457 2023-02-13 12:24:55,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201., pid=53, masterSystemTime=1676291095430 2023-02-13 12:24:55,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:24:55,459 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a3b734d5c30952e348ac86e3bc696201, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:55,459 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676291095459"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291095459"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291095459"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291095459"}]},"ts":"1676291095459"} 2023-02-13 12:24:55,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-13 12:24:55,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure a3b734d5c30952e348ac86e3bc696201, server=jenkins-hbase16.apache.org,38471,1676291011293 in 184 msec 2023-02-13 12:24:55,464 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 12:24:55,465 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a3b734d5c30952e348ac86e3bc696201, REOPEN/MOVE in 517 msec 2023-02-13 12:24:55,467 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 12:24:55,467 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 523 msec 2023-02-13 12:24:55,468 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 947 msec 2023-02-13 12:24:55,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 12:24:55,471 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(278): alter status finished 2023-02-13 12:24:55,471 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-13 12:24:55,473 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:55,473 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:55,474 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:55,474 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291695474 2023-02-13 12:24:55,475 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:24:55,475 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-13 12:24:55,475 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:55,476 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x67c65a76 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:24:55,514 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ce2eb83, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-13 12:24:55,514 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-13 12:24:55,515 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:24:55,516 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-13 12:24:55,518 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 44 msec 2023-02-13 12:24:55,581 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=237 (was 249), OpenFileDescriptor=598 (was 607), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=101 (was 122), ProcessCount=170 (was 170), AvailableMemoryMB=6015 (was 6061) 2023-02-13 12:24:55,590 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=237, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=101, ProcessCount=170, AvailableMemoryMB=6015 2023-02-13 12:24:55,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-13 12:24:55,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:55,594 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:24:55,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-13 12:24:55,594 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:24:55,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:55,595 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:55,596 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 empty. 2023-02-13 12:24:55,597 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:55,597 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-13 12:24:55,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:55,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:56,009 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 12:24:56,012 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => e8a189e2b6c4b0348126d69cf6d787a1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:24:56,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:56,426 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:56,426 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing e8a189e2b6c4b0348126d69cf6d787a1, disabling compactions & flushes 2023-02-13 12:24:56,427 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,427 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,427 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. after waiting 0 ms 2023-02-13 12:24:56,427 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,427 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,427 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for e8a189e2b6c4b0348126d69cf6d787a1: Waiting for close lock at 1676291096426Disabling compacts and flushes for region at 1676291096426Disabling writes for close at 1676291096427 (+1 ms)Writing region close event to WAL at 1676291096427Closed at 1676291096427 2023-02-13 12:24:56,432 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:24:56,434 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291096433"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291096433"}]},"ts":"1676291096433"} 2023-02-13 12:24:56,436 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:24:56,437 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:24:56,437 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291096437"}]},"ts":"1676291096437"} 2023-02-13 12:24:56,439 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-13 12:24:56,480 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, ASSIGN}] 2023-02-13 12:24:56,481 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, ASSIGN 2023-02-13 12:24:56,482 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:24:56,633 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:56,634 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291096633"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291096633"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291096633"}]},"ts":"1676291096633"} 2023-02-13 12:24:56,636 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:24:56,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:56,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e8a189e2b6c4b0348126d69cf6d787a1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:24:56,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:24:56,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,801 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,804 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:24:56,805 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:24:56,805 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e8a189e2b6c4b0348126d69cf6d787a1 columnFamilyName family1 2023-02-13 12:24:56,807 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(310): Store=e8a189e2b6c4b0348126d69cf6d787a1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:24:56,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:24:56,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:24:56,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e8a189e2b6c4b0348126d69cf6d787a1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72062720, jitterRate=0.07381820678710938}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:24:56,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e8a189e2b6c4b0348126d69cf6d787a1: Running coprocessor pre-open hook at 1676291096798Writing region info on filesystem at 1676291096798Initializing all the Stores at 1676291096800 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676291096800Cleaning up temporary data from old regions at 1676291096810 (+10 ms)Cleaning up detritus from prior splits at 1676291096811 (+1 ms)Running coprocessor post-open hooks at 1676291096819 (+8 ms)Region opened successfully at 1676291096819 2023-02-13 12:24:56,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., pid=57, masterSystemTime=1676291096789 2023-02-13 12:24:56,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:56,822 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:56,823 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291096822"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291096822"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291096822"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291096822"}]},"ts":"1676291096822"} 2023-02-13 12:24:56,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 12:24:56,827 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293 in 189 msec 2023-02-13 12:24:56,829 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-13 12:24:56,829 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, ASSIGN in 347 msec 2023-02-13 12:24:56,829 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:24:56,830 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291096829"}]},"ts":"1676291096829"} 2023-02-13 12:24:56,831 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-13 12:24:56,839 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:24:56,840 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.2470 sec 2023-02-13 12:24:57,359 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:57,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-13 12:24:57,707 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-13 12:24:57,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:57,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,735 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:57,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:57,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:24:57,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:24:57,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,771 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:57,771 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:57,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:57,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:57,772 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:57,772 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:57,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:57,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:57,773 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:24:57,773 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:57,773 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:24:57,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:57,774 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:57,774 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:57,774 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:24:57,774 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:24:58,196 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511304a46fde1afa64a1284ea86d9823b5945 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511304a46fde1afa64a1284ea86d9823b5945 2023-02-13 12:24:58,600 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/39bc947a74a547f8ae21eda0a22db74e 2023-02-13 12:24:58,607 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/39bc947a74a547f8ae21eda0a22db74e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e 2023-02-13 12:24:58,612 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-13 12:24:58,613 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 839ms, sequenceid=5, compaction requested=false 2023-02-13 12:24:58,614 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:24:58,614 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:58,614 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:58,614 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:58,614 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,614 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:24:58,614 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:24:58,646 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,646 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:58,647 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,647 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:58,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:24:58,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d4c00[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:58,648 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:24:58,648 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,658 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,658 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:24:58,658 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,659 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:58,658 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:24:58,659 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:58,666 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:24:58,666 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,666 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:58,667 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:58,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:24:58,669 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:24:58,669 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:58,669 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:58,669 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,669 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:58,679 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,679 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:58,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,679 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:58,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:58,680 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:58,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,681 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:58,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,699 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,714 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:58,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:58,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:58,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:58,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:58,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:58,715 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:58,715 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:58,815 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:58,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:58,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:58,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:58,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,821 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:58,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:58,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:24:58,821 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:24:58,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,833 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:58,833 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:58,834 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:58,834 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:58,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,835 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:24:58,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:58,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:24:58,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:58,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:24:58,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:58,835 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:58,835 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:58,835 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:24:58,835 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:24:59,253 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112818116fe246ee482baf3a67d644d7e527 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818116fe246ee482baf3a67d644d7e527 2023-02-13 12:24:59,660 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b 2023-02-13 12:24:59,665 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b 2023-02-13 12:24:59,668 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:24:59,669 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 834ms, sequenceid=9, compaction requested=false 2023-02-13 12:24:59,669 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:24:59,669 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:59,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:24:59,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:24:59,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:24:59,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:24:59,705 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,705 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:59,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:59,705 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,706 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:24:59,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:59,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:59,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:59,709 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:24:59,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18a1737b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:24:59,710 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:24:59,710 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,717 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,717 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:24:59,717 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:24:59,717 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:24:59,718 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,718 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:24:59,725 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,725 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:24:59,725 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:59,725 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:24:59,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:59,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:59,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:59,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:24:59,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:24:59,735 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:24:59,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:24:59,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:24:59,735 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,735 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:24:59,746 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,746 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,746 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:59,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:24:59,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:24:59,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:24:59,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:59,748 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:59,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:24:59,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,751 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:24:59,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,763 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:24:59,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:59,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:59,763 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:59,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:59,764 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:24:59,764 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:24:59,764 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:24:59,764 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,764 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:24:59,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:24:59,864 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:24:59,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:24:59,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:24:59,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:24:59,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,879 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:24:59,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:59,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:24:59,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:24:59,881 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,881 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,891 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,891 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:24:59,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:24:59,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:24:59,892 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:24:59,892 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:24:59,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,892 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:24:59,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:24:59,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:24:59,893 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:24:59,893 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:24:59,893 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:24:59,893 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:24:59,893 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:24:59,893 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:24:59,894 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:24:59,894 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:00,317 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201b4b12be6accc4b8782fb7f8eebbaaa8d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4b12be6accc4b8782fb7f8eebbaaa8d 2023-02-13 12:25:00,723 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/6e14bfabdcf04a1291930a99bf9a6b31 2023-02-13 12:25:00,729 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/6e14bfabdcf04a1291930a99bf9a6b31 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31 2023-02-13 12:25:00,735 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31, entries=1, sequenceid=13, filesize=5.0 K 2023-02-13 12:25:00,736 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 842ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:00,736 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:00,736 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:00,736 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:00,736 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:00,736 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,736 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:00,736 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:00,771 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,771 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:00,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:00,772 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,772 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:00,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:00,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:00,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:00,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3718ca98[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:00,777 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:00,777 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,788 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,788 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:00,789 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:00,789 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:00,789 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,789 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:00,800 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,800 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:00,800 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:00,800 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:00,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:00,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:00,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:00,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:00,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:00,811 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:00,811 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:00,811 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:00,811 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,812 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:00,821 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,821 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,821 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:00,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:00,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:00,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:00,822 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:00,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:00,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,825 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,838 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:00,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:00,838 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,839 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:00,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:00,838 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:00,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:00,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:00,839 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,839 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:00,839 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,839 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:00,940 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:00,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:00,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:00,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:00,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,956 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:00,956 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:00,956 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:00,956 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:00,957 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,957 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,975 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:00,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:00,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:00,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:00,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:00,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:00,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:00,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:00,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:00,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:00,978 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:00,978 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:00,978 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:01,399 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a5a01c2ce91e4fa3b7d53df914f4e834 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5a01c2ce91e4fa3b7d53df914f4e834 2023-02-13 12:25:01,808 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/cd413b764e3241f699fa49364001eb67 2023-02-13 12:25:01,818 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/cd413b764e3241f699fa49364001eb67 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67 2023-02-13 12:25:01,825 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67, entries=1, sequenceid=17, filesize=5.0 K 2023-02-13 12:25:01,826 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 848ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:01,826 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:01,826 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:01,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:01,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:01,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:01,826 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:01,879 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,879 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:01,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:01,880 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,880 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:01,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:01,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:01,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:01,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:01,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fd580bd[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:01,885 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:01,885 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,900 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,901 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:01,901 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:01,901 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:01,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,901 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:01,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:01,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:01,913 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:01,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:01,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:01,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:01,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:01,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:01,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:01,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:01,923 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:01,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:01,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:01,923 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,924 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:01,933 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,933 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,933 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:01,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:01,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:01,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:01,934 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:01,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,937 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:01,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,950 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:01,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:01,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:01,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:01,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:01,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:01,951 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:01,951 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:01,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:01,951 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:01,952 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:01,952 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,052 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:02,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:02,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:02,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:02,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,068 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:02,068 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:02,068 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:02,068 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:02,069 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,069 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,079 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:02,080 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:02,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:02,081 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,081 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:02,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,082 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:02,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:02,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,085 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:02,085 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:02,085 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:02,085 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:02,085 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:02,085 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:02,085 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:02,086 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:02,511 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d1c4a6d821f54518a54caf6fc054b5bb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d1c4a6d821f54518a54caf6fc054b5bb 2023-02-13 12:25:02,918 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/d6274595215b425389fc576453102d73 2023-02-13 12:25:02,928 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/d6274595215b425389fc576453102d73 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73 2023-02-13 12:25:02,934 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73, entries=1, sequenceid=21, filesize=5.0 K 2023-02-13 12:25:02,935 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 849ms, sequenceid=21, compaction requested=true 2023-02-13 12:25:02,935 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:02,935 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:02,935 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:02,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:02,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:02,936 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:02,971 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,971 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:02,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:02,972 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,972 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:02,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:02,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:02,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:02,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:02,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5728c0b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:02,977 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:02,977 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,988 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:02,988 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:02,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:02,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:02,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:02,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:03,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:03,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:03,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:03,000 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:03,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:03,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:03,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:03,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:03,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:03,011 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:03,011 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:03,011 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:03,011 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:03,011 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,011 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:03,021 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,021 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:03,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,021 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:03,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:03,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:03,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:03,021 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:03,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:03,023 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:03,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:03,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:03,033 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:03,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:03,034 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:03,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:03,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,034 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,034 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:03,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:03,134 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:03,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:03,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:03,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:03,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,141 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:03,141 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:03,141 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:03,141 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:03,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,150 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,150 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:03,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:03,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:03,150 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:03,150 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:03,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:03,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:03,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:03,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:03,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:03,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:03,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:03,152 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:03,152 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:03,152 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:03,152 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:03,571 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231afd3d49ede92468499417dd67aefa28f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231afd3d49ede92468499417dd67aefa28f 2023-02-13 12:25:03,982 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/0232dd1eec124deb80d6f3e80c8add74 2023-02-13 12:25:03,994 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/0232dd1eec124deb80d6f3e80c8add74 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74 2023-02-13 12:25:03,999 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74, entries=1, sequenceid=25, filesize=5.0 K 2023-02-13 12:25:04,000 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 848ms, sequenceid=25, compaction requested=true 2023-02-13 12:25:04,000 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:04,000 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:04,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:04,001 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:04,001 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,001 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:04,001 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:04,038 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,038 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:04,038 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,038 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:04,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:04,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:04,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:04,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@165b0d75[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:04,041 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:04,041 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,050 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:04,050 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:04,050 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:04,050 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,050 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:04,058 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,058 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:04,058 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:04,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:04,058 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:04,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:04,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:04,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:04,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:04,065 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:04,065 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:04,065 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:04,065 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,065 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:04,091 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,091 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:04,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,092 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:04,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:04,093 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:04,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:04,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,097 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,133 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:04,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:04,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:04,133 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:04,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:04,134 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:04,134 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:04,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:04,134 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:04,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:04,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:04,235 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:04,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:04,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:04,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:04,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,245 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:04,245 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:04,245 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:04,245 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:04,245 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,245 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,258 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:04,258 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:04,258 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:04,259 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:04,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,260 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:04,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:04,260 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:04,260 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:04,260 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:04,260 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:04,260 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:04,260 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:04,260 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:04,261 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:04,686 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101581fff08b0614fb0af41f56d4946be2d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101581fff08b0614fb0af41f56d4946be2d 2023-02-13 12:25:05,094 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/fccaa96e64aa4bfbba0c6badc1db4718 2023-02-13 12:25:05,105 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/fccaa96e64aa4bfbba0c6badc1db4718 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718 2023-02-13 12:25:05,111 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718, entries=1, sequenceid=29, filesize=5.0 K 2023-02-13 12:25:05,112 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 851ms, sequenceid=29, compaction requested=true 2023-02-13 12:25:05,112 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:05,112 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:05,112 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:05,112 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:05,112 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,113 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:05,113 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:05,146 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,146 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:05,147 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,147 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:05,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:05,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:05,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:05,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:05,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6766592c[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:05,151 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:05,152 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,163 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,163 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:05,163 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:05,163 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:05,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:05,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:05,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:05,171 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:05,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:05,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:05,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:05,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:05,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:05,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:05,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:05,173 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:05,173 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:05,173 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:05,173 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,173 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:05,183 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:05,183 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,183 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:05,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:05,183 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:05,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:05,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:05,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,185 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:05,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:05,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:05,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:05,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:05,196 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,196 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:05,197 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:05,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:05,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:05,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:05,297 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:05,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:05,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:05,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:05,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,309 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:05,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:05,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:05,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:05,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,310 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,316 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:05,316 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:05,317 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:05,317 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:05,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:05,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,318 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:05,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:05,319 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:05,319 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:05,319 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:05,319 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:05,319 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:05,319 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:05,319 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:05,319 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:05,741 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c8f33708fc0a41af8376c1fd6034912c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c8f33708fc0a41af8376c1fd6034912c 2023-02-13 12:25:06,150 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/141c811727c74ab4a3c2fb421ae2d372 2023-02-13 12:25:06,161 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/141c811727c74ab4a3c2fb421ae2d372 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372 2023-02-13 12:25:06,165 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372, entries=1, sequenceid=33, filesize=5.0 K 2023-02-13 12:25:06,166 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 847ms, sequenceid=33, compaction requested=true 2023-02-13 12:25:06,166 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:06,166 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:06,166 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:06,166 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:06,166 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,167 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:06,167 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:06,204 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,204 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:06,205 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:06,205 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:06,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:06,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2475fd15[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:06,206 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:06,206 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,216 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,216 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:06,217 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:06,217 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,217 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:06,217 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:06,225 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:06,225 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:06,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,225 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:06,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:06,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:06,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:06,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:06,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:06,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:06,228 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:06,228 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:06,228 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:06,228 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,228 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:06,237 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,237 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,238 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:06,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:06,238 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:06,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,240 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:06,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,249 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:06,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:06,250 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:06,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,250 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:06,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,250 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:06,251 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,251 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,351 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:06,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:06,363 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:06,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:06,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,367 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:06,367 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:06,367 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:06,367 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:06,368 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,368 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,375 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:06,375 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:06,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:06,375 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:06,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,376 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:06,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,378 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:06,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:06,378 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:06,378 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:06,378 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:06,379 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:06,379 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:06,379 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:06,379 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:06,379 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:06,797 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ce518ed5aef2424bbaad59d2a763342e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907ce518ed5aef2424bbaad59d2a763342e 2023-02-13 12:25:07,203 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/d4b58136081c4c62973bb3b058060006 2023-02-13 12:25:07,214 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/d4b58136081c4c62973bb3b058060006 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006 2023-02-13 12:25:07,218 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006, entries=1, sequenceid=37, filesize=5.0 K 2023-02-13 12:25:07,219 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 840ms, sequenceid=37, compaction requested=true 2023-02-13 12:25:07,219 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:07,219 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:07,219 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:07,219 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:07,219 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,219 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:07,219 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:07,268 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,268 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:07,268 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:07,268 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:07,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:07,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35763f44[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:07,271 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:07,272 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,291 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,292 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:07,292 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:07,292 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:07,292 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,292 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:07,300 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:07,300 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:07,300 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:07,300 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:07,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:07,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:07,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:07,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:07,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:07,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:07,309 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:07,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:07,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:07,309 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,309 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:07,321 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,321 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:07,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,321 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:07,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:07,322 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,322 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:07,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,333 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:07,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:07,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:07,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,333 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:07,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:07,334 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:07,334 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:07,434 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:07,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:07,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:07,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:07,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,445 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:07,445 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:07,445 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-13 12:25:07,445 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-13 12:25:07,446 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,446 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,454 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:07,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:07,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:07,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,456 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:07,456 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:07,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,458 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-13 12:25:07,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:07,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-13 12:25:07,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:07,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-13 12:25:07,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:25:07,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:07,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:07,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. started... 2023-02-13 12:25:07,460 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e8a189e2b6c4b0348126d69cf6d787a1 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:25:07,889 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c6d7a3d5764f49698f1a71dbec589664 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c6d7a3d5764f49698f1a71dbec589664 2023-02-13 12:25:08,296 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/c3cb47adaf144eb3917c11b1f9e11239 2023-02-13 12:25:08,307 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/c3cb47adaf144eb3917c11b1f9e11239 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239 2023-02-13 12:25:08,312 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239, entries=1, sequenceid=41, filesize=5.0 K 2023-02-13 12:25:08,313 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for e8a189e2b6c4b0348126d69cf6d787a1 in 853ms, sequenceid=41, compaction requested=true 2023-02-13 12:25:08,313 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:08,314 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:08,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:25:08,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:25:08,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-13 12:25:08,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-13 12:25:08,346 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:08,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:08,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:08,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:08,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:08,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-13 12:25:08,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b08ac3a[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:08,350 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-13 12:25:08,350 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,358 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,358 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-13 12:25:08,358 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-13 12:25:08,358 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:08,358 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,359 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:08,366 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-13 12:25:08,366 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,366 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:08,367 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-13 12:25:08,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:08,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:08,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:08,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:08,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:08,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:08,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-13 12:25:08,374 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-13 12:25:08,374 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:08,374 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:08,374 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,374 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:08,383 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,383 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:08,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:08,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,383 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:08,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:08,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:08,385 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:08,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:08,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,388 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:08,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,399 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,400 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,400 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,400 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:08,399 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:08,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-13 12:25:08,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-13 12:25:08,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:08,400 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:08,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:08,400 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:08,401 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:08,401 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:08,400 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:08,401 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:08,401 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,501 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:08,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:08,505 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:08,507 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:08,509 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:08,510 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291708510 2023-02-13 12:25:08,510 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:08,512 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:25:08,514 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 12:25:08,514 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:08,514 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x36a6edbd to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:08,525 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30411696, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:08,525 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,388 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e0897d80c393466a99f36175d019b852 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103e0897d80c393466a99f36175d019b852 2023-02-13 12:25:09,390 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:09,403 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:09,404 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41150, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:09,412 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:25:09,413 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60840, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:25:09,423 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3781d71b4be04b65a3522be5aae4e259 first=Optional[r3] last=Optional[r6] 2023-02-13 12:25:09,424 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3781d71b4be04b65a3522be5aae4e259}] 2023-02-13 12:25:09,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3781d71b4be04b65a3522be5aae4e259 for inclusion in e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:09,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-13 12:25:09,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:25:09,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:09,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3781d71b4be04b65a3522be5aae4e259 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dkopsj8vmcfle95ifga52hs3f9oqhsn4jceshbp27rq2f0flmc8aulo5ro1t4fje/family1/3781d71b4be04b65a3522be5aae4e259 2023-02-13 12:25:09,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dkopsj8vmcfle95ifga52hs3f9oqhsn4jceshbp27rq2f0flmc8aulo5ro1t4fje/family1/3781d71b4be04b65a3522be5aae4e259 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_ 2023-02-13 12:25:09,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dkopsj8vmcfle95ifga52hs3f9oqhsn4jceshbp27rq2f0flmc8aulo5ro1t4fje/family1/3781d71b4be04b65a3522be5aae4e259 into e8a189e2b6c4b0348126d69cf6d787a1/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_ - updating store file list. 2023-02-13 12:25:09,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_ into e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:09,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dkopsj8vmcfle95ifga52hs3f9oqhsn4jceshbp27rq2f0flmc8aulo5ro1t4fje/family1/3781d71b4be04b65a3522be5aae4e259 into e8a189e2b6c4b0348126d69cf6d787a1/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_) 2023-02-13 12:25:09,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dkopsj8vmcfle95ifga52hs3f9oqhsn4jceshbp27rq2f0flmc8aulo5ro1t4fje/family1/3781d71b4be04b65a3522be5aae4e259 2023-02-13 12:25:09,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:25:09,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:25:09,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): e8a189e2b6c4b0348126d69cf6d787a1/family1 is initiating minor compaction 2023-02-13 12:25:09,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6be6ba8e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:09,442 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e8a189e2b6c4b0348126d69cf6d787a1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:09,442 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp, totalSize=50.3 K 2023-02-13 12:25:09,442 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting bf16a3c0f6364e94a2d2a1b3c29c5c6b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:25:09,442 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 6e14bfabdcf04a1291930a99bf9a6b31, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting cd413b764e3241f699fa49364001eb67, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting d6274595215b425389fc576453102d73, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 0232dd1eec124deb80d6f3e80c8add74, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting fccaa96e64aa4bfbba0c6badc1db4718, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 141c811727c74ab4a3c2fb421ae2d372, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting d4b58136081c4c62973bb3b058060006, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting c3cb47adaf144eb3917c11b1f9e11239, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:25:09,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting 53e9583ac6da42238748debbed5c8818_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:25:09,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:09,445 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:09,446 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d1c4a6d821f54518a54caf6fc054b5bb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d1c4a6d821f54518a54caf6fc054b5bb 2023-02-13 12:25:09,447 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231afd3d49ede92468499417dd67aefa28f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231afd3d49ede92468499417dd67aefa28f 2023-02-13 12:25:09,448 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101581fff08b0614fb0af41f56d4946be2d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101581fff08b0614fb0af41f56d4946be2d 2023-02-13 12:25:09,450 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c8f33708fc0a41af8376c1fd6034912c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c8f33708fc0a41af8376c1fd6034912c 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-13 12:25:09,450 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,450 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,451 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:09,464 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): e8a189e2b6c4b0348126d69cf6d787a1#family1#compaction#73 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:25:10,284 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/833787879043416a8fcf5ab3781f0b70 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/833787879043416a8fcf5ab3781f0b70 2023-02-13 12:25:10,287 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ac217eb4474c421fb94187d2f41aa3a7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205ac217eb4474c421fb94187d2f41aa3a7 2023-02-13 12:25:10,288 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:10,289 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in e8a189e2b6c4b0348126d69cf6d787a1/family1 of e8a189e2b6c4b0348126d69cf6d787a1 into 833787879043416a8fcf5ab3781f0b70(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:10,289 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:10,289 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676291109442; duration=0sec 2023-02-13 12:25:10,289 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:10,310 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bf2b789125948ff9f7e580002d17697 first=Optional[r00] last=Optional[r2] 2023-02-13 12:25:10,311 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bf2b789125948ff9f7e580002d17697}] 2023-02-13 12:25:10,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bf2b789125948ff9f7e580002d17697 for inclusion in e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:10,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-13 12:25:10,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:25:10,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:10,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bf2b789125948ff9f7e580002d17697 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jmco6na4ekn41h1igi1bfqhh8nfndgcuhk9tf5grrh010jr89b0ug1g2a0n9aev0/family1/1bf2b789125948ff9f7e580002d17697 2023-02-13 12:25:10,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jmco6na4ekn41h1igi1bfqhh8nfndgcuhk9tf5grrh010jr89b0ug1g2a0n9aev0/family1/1bf2b789125948ff9f7e580002d17697 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_ 2023-02-13 12:25:10,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jmco6na4ekn41h1igi1bfqhh8nfndgcuhk9tf5grrh010jr89b0ug1g2a0n9aev0/family1/1bf2b789125948ff9f7e580002d17697 into e8a189e2b6c4b0348126d69cf6d787a1/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_ - updating store file list. 2023-02-13 12:25:10,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_ into e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:10,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jmco6na4ekn41h1igi1bfqhh8nfndgcuhk9tf5grrh010jr89b0ug1g2a0n9aev0/family1/1bf2b789125948ff9f7e580002d17697 into e8a189e2b6c4b0348126d69cf6d787a1/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_) 2023-02-13 12:25:10,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__jmco6na4ekn41h1igi1bfqhh8nfndgcuhk9tf5grrh010jr89b0ug1g2a0n9aev0/family1/1bf2b789125948ff9f7e580002d17697 2023-02-13 12:25:10,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:25:10,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:25:10,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): e8a189e2b6c4b0348126d69cf6d787a1/family1 is initiating minor compaction (all files) 2023-02-13 12:25:10,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@33ed3822; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:10,324 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e8a189e2b6c4b0348126d69cf6d787a1/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:10,324 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/833787879043416a8fcf5ab3781f0b70, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp, totalSize=388.1 K 2023-02-13 12:25:10,325 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 39bc947a74a547f8ae21eda0a22db74e, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:25:10,325 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 833787879043416a8fcf5ab3781f0b70, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:25:10,325 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 38c1ea0301694cd8b18379a0acc20034_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836144 2023-02-13 12:25:10,326 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:10,326 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:10,327 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511304a46fde1afa64a1284ea86d9823b5945 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511304a46fde1afa64a1284ea86d9823b5945 2023-02-13 12:25:10,328 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4b12be6accc4b8782fb7f8eebbaaa8d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201b4b12be6accc4b8782fb7f8eebbaaa8d 2023-02-13 12:25:10,329 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5a01c2ce91e4fa3b7d53df914f4e834 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a5a01c2ce91e4fa3b7d53df914f4e834 2023-02-13 12:25:10,329 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-13 12:25:10,330 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-13 12:25:10,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:10,330 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:10,330 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:10,331 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:10,332 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8250 sec 2023-02-13 12:25:10,340 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): e8a189e2b6c4b0348126d69cf6d787a1#family1#compaction#74 average throughput is 29.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:25:10,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_] to archive 2023-02-13 12:25:10,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:10,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/bf16a3c0f6364e94a2d2a1b3c29c5c6b 2023-02-13 12:25:10,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/6e14bfabdcf04a1291930a99bf9a6b31 2023-02-13 12:25:10,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/cd413b764e3241f699fa49364001eb67 2023-02-13 12:25:10,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d6274595215b425389fc576453102d73 2023-02-13 12:25:10,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/0232dd1eec124deb80d6f3e80c8add74 2023-02-13 12:25:10,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/fccaa96e64aa4bfbba0c6badc1db4718 2023-02-13 12:25:10,386 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/141c811727c74ab4a3c2fb421ae2d372 2023-02-13 12:25:10,387 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d4b58136081c4c62973bb3b058060006 2023-02-13 12:25:10,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/c3cb47adaf144eb3917c11b1f9e11239 2023-02-13 12:25:10,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/53e9583ac6da42238748debbed5c8818_SeqId_44_ 2023-02-13 12:25:10,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 12:25:10,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:10,824 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 12:25:10,826 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-13 12:25:10,827 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 12:25:10,831 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 12:25:10,834 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE}] 2023-02-13 12:25:10,835 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE 2023-02-13 12:25:10,835 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:10,835 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291110835"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291110835"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291110835"}]},"ts":"1676291110835"} 2023-02-13 12:25:10,837 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:10,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:10,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e8a189e2b6c4b0348126d69cf6d787a1, disabling compactions & flushes 2023-02-13 12:25:10,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,566 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/.tmp/family1/d3915e84c77c4b0bb852c41749a63ad3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d3915e84c77c4b0bb852c41749a63ad3 2023-02-13 12:25:11,572 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in e8a189e2b6c4b0348126d69cf6d787a1/family1 of e8a189e2b6c4b0348126d69cf6d787a1 into d3915e84c77c4b0bb852c41749a63ad3(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:11,572 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:11,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,572 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1676291110324; duration=1sec 2023-02-13 12:25:11,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,572 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:11,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. after waiting 0 ms 2023-02-13 12:25:11,572 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,576 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/833787879043416a8fcf5ab3781f0b70, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_] to archive 2023-02-13 12:25:11,577 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:11,579 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/39bc947a74a547f8ae21eda0a22db74e 2023-02-13 12:25:11,581 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/833787879043416a8fcf5ab3781f0b70 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/833787879043416a8fcf5ab3781f0b70 2023-02-13 12:25:11,582 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/38c1ea0301694cd8b18379a0acc20034_SeqId_47_ 2023-02-13 12:25:11,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 12:25:11,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e8a189e2b6c4b0348126d69cf6d787a1: Waiting for close lock at 1676291110989Running coprocessor pre-close hooks at 1676291110989Disabling compacts and flushes for region at 1676291110989Disabling writes for close at 1676291111572 (+583 ms)Writing region close event to WAL at 1676291111585 (+13 ms)Running coprocessor post-close hooks at 1676291111589 (+4 ms)Closed at 1676291111590 (+1 ms) 2023-02-13 12:25:11,590 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e8a189e2b6c4b0348126d69cf6d787a1 to self. 2023-02-13 12:25:11,592 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,592 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=CLOSED 2023-02-13 12:25:11,593 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291111592"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291111592"}]},"ts":"1676291111592"} 2023-02-13 12:25:11,596 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-13 12:25:11,597 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293 in 757 msec 2023-02-13 12:25:11,597 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=true 2023-02-13 12:25:11,748 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:11,748 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291111748"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291111748"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291111748"}]},"ts":"1676291111748"} 2023-02-13 12:25:11,750 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:11,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e8a189e2b6c4b0348126d69cf6d787a1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:25:11,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:11,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,914 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,917 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:11,917 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:11,918 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e8a189e2b6c4b0348126d69cf6d787a1 columnFamilyName family1 2023-02-13 12:25:11,926 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d3915e84c77c4b0bb852c41749a63ad3 2023-02-13 12:25:11,926 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(310): Store=e8a189e2b6c4b0348126d69cf6d787a1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:11,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:11,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e8a189e2b6c4b0348126d69cf6d787a1; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71192932, jitterRate=0.06085735559463501}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:25:11,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e8a189e2b6c4b0348126d69cf6d787a1: Running coprocessor pre-open hook at 1676291111911Writing region info on filesystem at 1676291111911Initializing all the Stores at 1676291111913 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676291111913Cleaning up temporary data from old regions at 1676291111930 (+17 ms)Cleaning up detritus from prior splits at 1676291111931 (+1 ms)Running coprocessor post-open hooks at 1676291111936 (+5 ms)Region opened successfully at 1676291111936 2023-02-13 12:25:11,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., pid=63, masterSystemTime=1676291111902 2023-02-13 12:25:11,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:11,940 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:11,941 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291111940"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291111940"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291111940"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291111940"}]},"ts":"1676291111940"} 2023-02-13 12:25:11,945 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-13 12:25:11,945 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293 in 194 msec 2023-02-13 12:25:11,948 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-13 12:25:11,948 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE in 1.1110 sec 2023-02-13 12:25:11,952 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-13 12:25:11,952 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1180 sec 2023-02-13 12:25:11,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5400 sec 2023-02-13 12:25:11,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-13 12:25:11,955 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(278): alter status finished 2023-02-13 12:25:11,956 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:11,958 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:11,958 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:11,959 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:11,959 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291711959 2023-02-13 12:25:11,959 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:25:11,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 12:25:11,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:11,961 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x63fd4c1d to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:12,118 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@153e899f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:12,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,550 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289a8592d6b7234fea974b5580cabfae74 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511289a8592d6b7234fea974b5580cabfae74 2023-02-13 12:25:12,551 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:12,562 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:12,563 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:42250, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:12,569 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:25:12,571 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41290, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:25:12,581 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3087ef16367f4323bf046573f0ef8d54 first=Optional[r05] last=Optional[r8] 2023-02-13 12:25:12,581 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=51 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3087ef16367f4323bf046573f0ef8d54}] 2023-02-13 12:25:12,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3087ef16367f4323bf046573f0ef8d54 for inclusion in e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:12,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-13 12:25:12,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:25:12,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for e8a189e2b6c4b0348126d69cf6d787a1: 2023-02-13 12:25:12,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3087ef16367f4323bf046573f0ef8d54 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m6f9tv4ea7gld9raancdavioepbm589qqikblubkq4t83glbvp2e2h9t1ajgvb2b/family1/3087ef16367f4323bf046573f0ef8d54 2023-02-13 12:25:12,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m6f9tv4ea7gld9raancdavioepbm589qqikblubkq4t83glbvp2e2h9t1ajgvb2b/family1/3087ef16367f4323bf046573f0ef8d54 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/a11839acb0164019b8335860de7c6c3a_SeqId_53_ 2023-02-13 12:25:12,592 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m6f9tv4ea7gld9raancdavioepbm589qqikblubkq4t83glbvp2e2h9t1ajgvb2b/family1/3087ef16367f4323bf046573f0ef8d54 into e8a189e2b6c4b0348126d69cf6d787a1/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/a11839acb0164019b8335860de7c6c3a_SeqId_53_ - updating store file list. 2023-02-13 12:25:12,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/a11839acb0164019b8335860de7c6c3a_SeqId_53_ into e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:12,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m6f9tv4ea7gld9raancdavioepbm589qqikblubkq4t83glbvp2e2h9t1ajgvb2b/family1/3087ef16367f4323bf046573f0ef8d54 into e8a189e2b6c4b0348126d69cf6d787a1/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/a11839acb0164019b8335860de7c6c3a_SeqId_53_) 2023-02-13 12:25:12,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__m6f9tv4ea7gld9raancdavioepbm589qqikblubkq4t83glbvp2e2h9t1ajgvb2b/family1/3087ef16367f4323bf046573f0ef8d54 2023-02-13 12:25:12,598 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:12,598 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:12,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c6d7a3d5764f49698f1a71dbec589664 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c6d7a3d5764f49698f1a71dbec589664 2023-02-13 12:25:12,600 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818116fe246ee482baf3a67d644d7e527 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112818116fe246ee482baf3a67d644d7e527 2023-02-13 12:25:12,600 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-13 12:25:12,600 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 12:25:12,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:12,601 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:12,601 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:12,602 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:12,603 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 645 msec 2023-02-13 12:25:12,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-13 12:25:12,655 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:13,067 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 12:25:13,070 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-13 12:25:13,070 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-13 12:25:13,074 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-13 12:25:13,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE}] 2023-02-13 12:25:13,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE 2023-02-13 12:25:13,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:13,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291113078"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291113078"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291113078"}]},"ts":"1676291113078"} 2023-02-13 12:25:13,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:13,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e8a189e2b6c4b0348126d69cf6d787a1, disabling compactions & flushes 2023-02-13 12:25:13,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. after waiting 0 ms 2023-02-13 12:25:13,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-13 12:25:13,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e8a189e2b6c4b0348126d69cf6d787a1: Waiting for close lock at 1676291113233Running coprocessor pre-close hooks at 1676291113233Disabling compacts and flushes for region at 1676291113233Disabling writes for close at 1676291113233Writing region close event to WAL at 1676291113245 (+12 ms)Running coprocessor post-close hooks at 1676291113248 (+3 ms)Closed at 1676291113248 2023-02-13 12:25:13,248 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: e8a189e2b6c4b0348126d69cf6d787a1 to self. 2023-02-13 12:25:13,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,250 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=CLOSED 2023-02-13 12:25:13,250 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291113250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291113250"}]},"ts":"1676291113250"} 2023-02-13 12:25:13,252 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-13 12:25:13,252 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293 in 171 msec 2023-02-13 12:25:13,253 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=true 2023-02-13 12:25:13,403 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:13,403 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291113403"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291113403"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291113403"}]},"ts":"1676291113403"} 2023-02-13 12:25:13,407 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:13,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e8a189e2b6c4b0348126d69cf6d787a1, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:25:13,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:13,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,572 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,573 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:13,573 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1 2023-02-13 12:25:13,573 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e8a189e2b6c4b0348126d69cf6d787a1 columnFamilyName family1 2023-02-13 12:25:13,578 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/a11839acb0164019b8335860de7c6c3a_SeqId_53_ 2023-02-13 12:25:13,581 DEBUG [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/family1/d3915e84c77c4b0bb852c41749a63ad3 2023-02-13 12:25:13,581 INFO [StoreOpener-e8a189e2b6c4b0348126d69cf6d787a1-1] regionserver.HStore(310): Store=e8a189e2b6c4b0348126d69cf6d787a1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:13,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:25:13,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e8a189e2b6c4b0348126d69cf6d787a1; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59483405, jitterRate=-0.11362819373607635}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:25:13,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e8a189e2b6c4b0348126d69cf6d787a1: Running coprocessor pre-open hook at 1676291113571Writing region info on filesystem at 1676291113571Initializing all the Stores at 1676291113572 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1676291113572Cleaning up temporary data from old regions at 1676291113583 (+11 ms)Cleaning up detritus from prior splits at 1676291113584 (+1 ms)Running coprocessor post-open hooks at 1676291113586 (+2 ms)Region opened successfully at 1676291113586 2023-02-13 12:25:13,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1., pid=69, masterSystemTime=1676291113561 2023-02-13 12:25:13,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:25:13,588 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=e8a189e2b6c4b0348126d69cf6d787a1, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:13,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676291113588"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291113588"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291113588"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291113588"}]},"ts":"1676291113588"} 2023-02-13 12:25:13,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-13 12:25:13,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure e8a189e2b6c4b0348126d69cf6d787a1, server=jenkins-hbase16.apache.org,38471,1676291011293 in 182 msec 2023-02-13 12:25:13,592 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-13 12:25:13,592 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=e8a189e2b6c4b0348126d69cf6d787a1, REOPEN/MOVE in 514 msec 2023-02-13 12:25:13,594 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 12:25:13,594 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 518 msec 2023-02-13 12:25:13,595 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 940 msec 2023-02-13 12:25:13,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-13 12:25:13,597 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(278): alter status finished 2023-02-13 12:25:13,598 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-13 12:25:13,599 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:13,600 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:13,600 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:13,600 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291713600 2023-02-13 12:25:13,601 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:25:13,601 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-13 12:25:13,602 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:13,602 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x3155aca3 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:13,647 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@506c8f74, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-13 12:25:13,647 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-13 12:25:13,647 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:13,648 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-13 12:25:13,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 50 msec 2023-02-13 12:25:13,708 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=250 (was 237) - Thread LEAK? -, OpenFileDescriptor=636 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 101), ProcessCount=167 (was 170), AvailableMemoryMB=5999 (was 6015) 2023-02-13 12:25:13,718 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=250, OpenFileDescriptor=636, MaxFileDescriptor=60000, SystemLoadAverage=86, ProcessCount=167, AvailableMemoryMB=5998 2023-02-13 12:25:13,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 12:25:13,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:13,723 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:25:13,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-13 12:25:13,724 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:25:13,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:13,725 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:13,725 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 empty. 2023-02-13 12:25:13,726 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:13,726 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 12:25:13,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:14,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:14,134 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 12:25:14,136 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6a5176ae52d39cadd06991eaeab47681, NAME => 'testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:14,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:14,546 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:14,547 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing 6a5176ae52d39cadd06991eaeab47681, disabling compactions & flushes 2023-02-13 12:25:14,547 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,547 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,547 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. after waiting 0 ms 2023-02-13 12:25:14,547 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,547 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,547 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 6a5176ae52d39cadd06991eaeab47681: Waiting for close lock at 1676291114547Disabling compacts and flushes for region at 1676291114547Disabling writes for close at 1676291114547Writing region close event to WAL at 1676291114547Closed at 1676291114547 2023-02-13 12:25:14,562 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:25:14,564 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676291114563"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291114563"}]},"ts":"1676291114563"} 2023-02-13 12:25:14,565 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:25:14,565 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:25:14,565 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291114565"}]},"ts":"1676291114565"} 2023-02-13 12:25:14,566 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-13 12:25:14,612 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, ASSIGN}] 2023-02-13 12:25:14,614 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, ASSIGN 2023-02-13 12:25:14,614 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:14,765 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=6a5176ae52d39cadd06991eaeab47681, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:14,766 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676291114765"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291114765"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291114765"}]},"ts":"1676291114765"} 2023-02-13 12:25:14,767 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 6a5176ae52d39cadd06991eaeab47681, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:14,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:14,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a5176ae52d39cadd06991eaeab47681, NAME => 'testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:25:14,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-13 12:25:14,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-13 12:25:14,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:14,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,932 INFO [StoreOpener-6a5176ae52d39cadd06991eaeab47681-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,934 DEBUG [StoreOpener-6a5176ae52d39cadd06991eaeab47681-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1 2023-02-13 12:25:14,934 DEBUG [StoreOpener-6a5176ae52d39cadd06991eaeab47681-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1 2023-02-13 12:25:14,935 INFO [StoreOpener-6a5176ae52d39cadd06991eaeab47681-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a5176ae52d39cadd06991eaeab47681 columnFamilyName f1 2023-02-13 12:25:14,936 INFO [StoreOpener-6a5176ae52d39cadd06991eaeab47681-1] regionserver.HStore(310): Store=6a5176ae52d39cadd06991eaeab47681/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:14,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:14,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:14,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 6a5176ae52d39cadd06991eaeab47681; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64427224, jitterRate=-0.03995954990386963}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:25:14,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 6a5176ae52d39cadd06991eaeab47681: Running coprocessor pre-open hook at 1676291114931Writing region info on filesystem at 1676291114931Initializing all the Stores at 1676291114932 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291114932Cleaning up temporary data from old regions at 1676291114938 (+6 ms)Cleaning up detritus from prior splits at 1676291114938Running coprocessor post-open hooks at 1676291114942 (+4 ms)Region opened successfully at 1676291114942 2023-02-13 12:25:14,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681., pid=73, masterSystemTime=1676291114919 2023-02-13 12:25:14,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:14,945 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=6a5176ae52d39cadd06991eaeab47681, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:14,945 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676291114945"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291114945"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291114945"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291114945"}]},"ts":"1676291114945"} 2023-02-13 12:25:14,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 12:25:14,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 6a5176ae52d39cadd06991eaeab47681, server=jenkins-hbase16.apache.org,38471,1676291011293 in 180 msec 2023-02-13 12:25:14,950 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-13 12:25:14,950 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, ASSIGN in 336 msec 2023-02-13 12:25:14,951 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:25:14,951 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291114951"}]},"ts":"1676291114951"} 2023-02-13 12:25:14,952 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-13 12:25:14,963 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:25:14,964 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.2420 sec 2023-02-13 12:25:15,363 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-13 12:25:15,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-13 12:25:15,832 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-13 12:25:15,832 DEBUG [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-13 12:25:15,836 INFO [Listener at localhost.localdomain/33629] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 12:25:15,840 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-13 12:25:15,841 INFO [Listener at localhost.localdomain/33629] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 12:25:15,841 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-13 12:25:15,847 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2744): Flushing 6a5176ae52d39cadd06991eaeab47681 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 12:25:16,273 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133b9e46a74fc74b3d9ae9df488d63091b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133b9e46a74fc74b3d9ae9df488d63091b 2023-02-13 12:25:16,641 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3155aca3 to 127.0.0.1:64508 2023-02-13 12:25:16,642 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,642 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:25:16,642 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x63fd4c1d to 127.0.0.1:64508 2023-02-13 12:25:16,642 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,642 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:25:16,642 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36a6edbd to 127.0.0.1:64508 2023-02-13 12:25:16,642 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,642 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x67c65a76 to 127.0.0.1:64508 2023-02-13 12:25:16,642 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,642 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:25:16,643 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3a57175a to 127.0.0.1:64508 2023-02-13 12:25:16,643 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,643 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:25:16,643 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x501ae067 to 127.0.0.1:64508 2023-02-13 12:25:16,643 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:16,681 INFO [Listener at localhost.localdomain/33629] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp/f1/c70f18cb2ff148ae9169a64845d17efb 2023-02-13 12:25:16,689 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp/f1/c70f18cb2ff148ae9169a64845d17efb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb 2023-02-13 12:25:16,696 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb, entries=1, sequenceid=5, filesize=4.9 K 2023-02-13 12:25:16,697 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 6a5176ae52d39cadd06991eaeab47681 in 850ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:16,698 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2445): Flush status journal for 6a5176ae52d39cadd06991eaeab47681: 2023-02-13 12:25:16,700 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2744): Flushing 6a5176ae52d39cadd06991eaeab47681 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-13 12:25:17,120 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213a0ac665f5ccd480f9f770cc932c8e226 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a0ac665f5ccd480f9f770cc932c8e226 2023-02-13 12:25:17,145 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:25:17,526 INFO [Listener at localhost.localdomain/33629] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc 2023-02-13 12:25:17,538 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc 2023-02-13 12:25:17,543 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc, entries=1, sequenceid=9, filesize=4.9 K 2023-02-13 12:25:17,544 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 6a5176ae52d39cadd06991eaeab47681 in 844ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:17,544 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2445): Flush status journal for 6a5176ae52d39cadd06991eaeab47681: 2023-02-13 12:25:17,546 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-13 12:25:17,547 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 12:25:17,548 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 12:25:17,548 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291717548 2023-02-13 12:25:17,548 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:17,549 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:25:17,549 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-13 12:25:17,549 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-13 12:25:17,549 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x352cbd47 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:17,588 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10f83b9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:17,589 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 12:25:17,598 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,598 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,649 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,649 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,699 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,700 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,750 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,750 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,801 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,801 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,852 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,853 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,904 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,904 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,956 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:17,956 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,008 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,008 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,059 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,060 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,111 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,111 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,162 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,162 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,214 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,214 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,266 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,266 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,318 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,318 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,370 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,370 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,422 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,422 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,433 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302131c90efb54a8042b083567d6fec694df9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302131c90efb54a8042b083567d6fec694df9 2023-02-13 12:25:18,434 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 12:25:18,446 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:18,446 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:42260, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:18,453 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:25:18,455 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41302, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:25:18,464 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/fc2bd685fe274b3785db32d2a9f00638 first=Optional[r1] last=Optional[r2] 2023-02-13 12:25:18,464 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{f1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/fc2bd685fe274b3785db32d2a9f00638}] 2023-02-13 12:25:18,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/fc2bd685fe274b3785db32d2a9f00638 for inclusion in 6a5176ae52d39cadd06991eaeab47681/f1 2023-02-13 12:25:18,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-13 12:25:18,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:25:18,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 6a5176ae52d39cadd06991eaeab47681: 2023-02-13 12:25:18,473 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,473 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/f1/fc2bd685fe274b3785db32d2a9f00638 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testGetAfterCompaction__1sovgpn1e988jiobcd0h6rc1m6vptpfibgnqhqobn1lku0q9ge7fa44ol76mopp1/f1/fc2bd685fe274b3785db32d2a9f00638 2023-02-13 12:25:18,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testGetAfterCompaction__1sovgpn1e988jiobcd0h6rc1m6vptpfibgnqhqobn1lku0q9ge7fa44ol76mopp1/f1/fc2bd685fe274b3785db32d2a9f00638 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_ 2023-02-13 12:25:18,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testGetAfterCompaction__1sovgpn1e988jiobcd0h6rc1m6vptpfibgnqhqobn1lku0q9ge7fa44ol76mopp1/f1/fc2bd685fe274b3785db32d2a9f00638 into 6a5176ae52d39cadd06991eaeab47681/f1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_ - updating store file list. 2023-02-13 12:25:18,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_ into 6a5176ae52d39cadd06991eaeab47681/f1 2023-02-13 12:25:18,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testGetAfterCompaction__1sovgpn1e988jiobcd0h6rc1m6vptpfibgnqhqobn1lku0q9ge7fa44ol76mopp1/f1/fc2bd685fe274b3785db32d2a9f00638 into 6a5176ae52d39cadd06991eaeab47681/f1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_) 2023-02-13 12:25:18,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testGetAfterCompaction__1sovgpn1e988jiobcd0h6rc1m6vptpfibgnqhqobn1lku0q9ge7fa44ol76mopp1/f1/fc2bd685fe274b3785db32d2a9f00638 2023-02-13 12:25:18,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(1912): 6a5176ae52d39cadd06991eaeab47681/f1 is initiating minor compaction 2023-02-13 12:25:18,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4bfb9310; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:18,483 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a5176ae52d39cadd06991eaeab47681/f1 in testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:18,483 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp, totalSize=10.0 K 2023-02-13 12:25:18,483 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 5ff76b0c5e5449b5b2a15a60ab6bcefc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:25:18,483 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting cfe6be19d5c94e89b6020c7d29789130_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:25:18,486 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testGetAfterCompaction' column='f1' 2023-02-13 12:25:18,486 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:18,488 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133b9e46a74fc74b3d9ae9df488d63091b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302133b9e46a74fc74b3d9ae9df488d63091b 2023-02-13 12:25:18,490 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a0ac665f5ccd480f9f770cc932c8e226 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230213a0ac665f5ccd480f9f770cc932c8e226 2023-02-13 12:25:18,490 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-13 12:25:18,490 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 12:25:18,490 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testGetAfterCompaction' and column='f1' 2023-02-13 12:25:18,490 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-13 12:25:18,490 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:25:18,492 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-13 12:25:18,493 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 946 msec 2023-02-13 12:25:18,495 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6a5176ae52d39cadd06991eaeab47681#f1#compaction#77 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 65.38 MB/second 2023-02-13 12:25:18,524 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,524 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,524 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,529 DEBUG [Listener at localhost.localdomain/33629] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 12:25:18,533 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:42264, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 12:25:18,534 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38471] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:18,534 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-13 12:25:18,534 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-13 12:25:18,535 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38471] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-13 12:25:18,535 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38471] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. because compaction request was cancelled 2023-02-13 12:25:18,587 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,587 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,587 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,639 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,639 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,639 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,690 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,691 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,691 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,743 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,743 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,743 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,795 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,795 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,795 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,847 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,848 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,848 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,899 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,899 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,899 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:18,951 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,951 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:18,951 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,002 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,003 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,003 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,054 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,054 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,055 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,106 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,107 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,107 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,158 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,159 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,159 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,211 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,211 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,211 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,263 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,263 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,263 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,315 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291117525; access_time=1676291116702; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,315 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,315 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676291118421; access_time=1676291117604; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-13 12:25:19,317 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/.tmp/f1/c1eb328f344a4d0ca71d833902974a4d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c1eb328f344a4d0ca71d833902974a4d 2023-02-13 12:25:19,322 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 6a5176ae52d39cadd06991eaeab47681/f1 of 6a5176ae52d39cadd06991eaeab47681 into c1eb328f344a4d0ca71d833902974a4d(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:19,322 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a5176ae52d39cadd06991eaeab47681: 2023-02-13 12:25:19,323 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1676291118482; duration=0sec 2023-02-13 12:25:19,323 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:19,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_] to archive 2023-02-13 12:25:19,362 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:19,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/5ff76b0c5e5449b5b2a15a60ab6bcefc 2023-02-13 12:25:19,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/cfe6be19d5c94e89b6020c7d29789130_SeqId_12_ 2023-02-13 12:25:19,366 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c1eb328f344a4d0ca71d833902974a4d; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676291119305; access_time=1676291118490; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,366 INFO [Listener at localhost.localdomain/33629] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676291116681; access_time=1676291115851; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-13 12:25:19,491 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-13 12:25:19,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testGetAfterCompaction 2023-02-13 12:25:19,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,502 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291119502"}]},"ts":"1676291119502"} 2023-02-13 12:25:19,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 12:25:19,503 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-13 12:25:19,545 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-13 12:25:19,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, UNASSIGN}] 2023-02-13 12:25:19,549 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, UNASSIGN 2023-02-13 12:25:19,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6a5176ae52d39cadd06991eaeab47681, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:19,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676291119550"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291119550"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291119550"}]},"ts":"1676291119550"} 2023-02-13 12:25:19,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 6a5176ae52d39cadd06991eaeab47681, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:19,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 12:25:19,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:19,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 6a5176ae52d39cadd06991eaeab47681, disabling compactions & flushes 2023-02-13 12:25:19,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:19,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:19,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. after waiting 0 ms 2023-02-13 12:25:19,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:19,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 12:25:19,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-13 12:25:19,718 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681. 2023-02-13 12:25:19,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 6a5176ae52d39cadd06991eaeab47681: Waiting for close lock at 1676291119707Running coprocessor pre-close hooks at 1676291119707Disabling compacts and flushes for region at 1676291119707Disabling writes for close at 1676291119707Writing region close event to WAL at 1676291119709 (+2 ms)Running coprocessor post-close hooks at 1676291119717 (+8 ms)Closed at 1676291119718 (+1 ms) 2023-02-13 12:25:19,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:19,720 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6a5176ae52d39cadd06991eaeab47681, regionState=CLOSED 2023-02-13 12:25:19,721 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676291119720"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291119720"}]},"ts":"1676291119720"} 2023-02-13 12:25:19,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-13 12:25:19,723 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 6a5176ae52d39cadd06991eaeab47681, server=jenkins-hbase16.apache.org,38471,1676291011293 in 169 msec 2023-02-13 12:25:19,724 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-13 12:25:19,725 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=6a5176ae52d39cadd06991eaeab47681, UNASSIGN in 175 msec 2023-02-13 12:25:19,725 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291119725"}]},"ts":"1676291119725"} 2023-02-13 12:25:19,726 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-13 12:25:19,732 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-13 12:25:19,734 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 237 msec 2023-02-13 12:25:19,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-13 12:25:19,807 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-13 12:25:19,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testGetAfterCompaction 2023-02-13 12:25:19,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,822 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,823 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 12:25:19,826 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:19,829 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1, FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/recovered.edits] 2023-02-13 12:25:19,832 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c1eb328f344a4d0ca71d833902974a4d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c1eb328f344a4d0ca71d833902974a4d 2023-02-13 12:25:19,834 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/f1/c70f18cb2ff148ae9169a64845d17efb 2023-02-13 12:25:19,837 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/recovered.edits/15.seqid to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681/recovered.edits/15.seqid 2023-02-13 12:25:19,837 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testGetAfterCompaction/6a5176ae52d39cadd06991eaeab47681 2023-02-13 12:25:19,837 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-13 12:25:19,838 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 12:25:19,839 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-13 12:25:19,842 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302131c90efb54a8042b083567d6fec694df9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302131c90efb54a8042b083567d6fec694df9 2023-02-13 12:25:19,842 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-13 12:25:19,844 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,868 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-13 12:25:19,871 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-13 12:25:19,872 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,872 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-13 12:25:19,872 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676291119872"}]},"ts":"9223372036854775807"} 2023-02-13 12:25:19,873 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 12:25:19,874 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6a5176ae52d39cadd06991eaeab47681, NAME => 'testGetAfterCompaction,,1676291113721.6a5176ae52d39cadd06991eaeab47681.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 12:25:19,874 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-13 12:25:19,874 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676291119874"}]},"ts":"9223372036854775807"} 2023-02-13 12:25:19,875 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-13 12:25:19,883 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-13 12:25:19,884 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 67 msec 2023-02-13 12:25:19,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-13 12:25:19,926 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-13 12:25:19,944 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=234 (was 250), OpenFileDescriptor=589 (was 636), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=79 (was 86), ProcessCount=167 (was 167), AvailableMemoryMB=5977 (was 5998) 2023-02-13 12:25:19,956 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=234, OpenFileDescriptor=589, MaxFileDescriptor=60000, SystemLoadAverage=79, ProcessCount=167, AvailableMemoryMB=5977 2023-02-13 12:25:19,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 12:25:19,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-13 12:25:19,961 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:25:19,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-13 12:25:19,962 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:25:19,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:19,964 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:19,964 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:19,964 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e empty. 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d empty. 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff empty. 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:19,965 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:19,965 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-13 12:25:20,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:20,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:20,373 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 12:25:20,375 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 31a6bfc09681f3a11506a4775f610e4d, NAME => 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:20,376 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 534c6c8744730b7c32dbd5f29d42ddff, NAME => 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:20,376 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5d424be18d07d0a984d49d2450dc4b7e, NAME => 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 5d424be18d07d0a984d49d2450dc4b7e, disabling compactions & flushes 2023-02-13 12:25:20,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. after waiting 0 ms 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:20,393 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:20,393 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 5d424be18d07d0a984d49d2450dc4b7e: Waiting for close lock at 1676291120393Disabling compacts and flushes for region at 1676291120393Disabling writes for close at 1676291120393Writing region close event to WAL at 1676291120393Closed at 1676291120393 2023-02-13 12:25:20,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:20,791 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:20,791 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:20,792 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 534c6c8744730b7c32dbd5f29d42ddff, disabling compactions & flushes 2023-02-13 12:25:20,792 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 31a6bfc09681f3a11506a4775f610e4d, disabling compactions & flushes 2023-02-13 12:25:20,792 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:20,792 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:20,792 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:20,792 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:20,793 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. after waiting 0 ms 2023-02-13 12:25:20,793 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. after waiting 0 ms 2023-02-13 12:25:20,793 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:20,793 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:20,793 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:20,793 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:20,794 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 534c6c8744730b7c32dbd5f29d42ddff: Waiting for close lock at 1676291120792Disabling compacts and flushes for region at 1676291120792Disabling writes for close at 1676291120793 (+1 ms)Writing region close event to WAL at 1676291120793Closed at 1676291120793 2023-02-13 12:25:20,794 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 31a6bfc09681f3a11506a4775f610e4d: Waiting for close lock at 1676291120792Disabling compacts and flushes for region at 1676291120792Disabling writes for close at 1676291120793 (+1 ms)Writing region close event to WAL at 1676291120793Closed at 1676291120793 2023-02-13 12:25:20,797 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:25:20,798 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291120798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291120798"}]},"ts":"1676291120798"} 2023-02-13 12:25:20,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291120798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291120798"}]},"ts":"1676291120798"} 2023-02-13 12:25:20,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291120798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291120798"}]},"ts":"1676291120798"} 2023-02-13 12:25:20,802 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 12:25:20,803 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:25:20,803 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291120803"}]},"ts":"1676291120803"} 2023-02-13 12:25:20,805 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 12:25:20,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN}] 2023-02-13 12:25:20,851 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN 2023-02-13 12:25:20,851 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN 2023-02-13 12:25:20,852 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN 2023-02-13 12:25:20,853 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:20,853 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:20,853 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:21,007 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,007 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,008 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291121007"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291121007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291121007"}]},"ts":"1676291121007"} 2023-02-13 12:25:21,008 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291121007"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291121007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291121007"}]},"ts":"1676291121007"} 2023-02-13 12:25:21,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291121007"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291121007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291121007"}]},"ts":"1676291121007"} 2023-02-13 12:25:21,012 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:21,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:21,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:21,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:21,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:21,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 534c6c8744730b7c32dbd5f29d42ddff, NAME => 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 12:25:21,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:21,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,174 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,176 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:21,176 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:21,177 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 534c6c8744730b7c32dbd5f29d42ddff columnFamilyName family1 2023-02-13 12:25:21,179 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(310): Store=534c6c8744730b7c32dbd5f29d42ddff/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,179 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,181 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2 2023-02-13 12:25:21,181 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2 2023-02-13 12:25:21,181 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 534c6c8744730b7c32dbd5f29d42ddff columnFamilyName family2 2023-02-13 12:25:21,183 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(310): Store=534c6c8744730b7c32dbd5f29d42ddff/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:21,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:21,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:21,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 534c6c8744730b7c32dbd5f29d42ddff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66730156, jitterRate=-0.005643188953399658}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:21,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 534c6c8744730b7c32dbd5f29d42ddff: Running coprocessor pre-open hook at 1676291121172Writing region info on filesystem at 1676291121172Initializing all the Stores at 1676291121173 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121173Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121174 (+1 ms)Cleaning up temporary data from old regions at 1676291121185 (+11 ms)Cleaning up detritus from prior splits at 1676291121185Running coprocessor post-open hooks at 1676291121188 (+3 ms)Region opened successfully at 1676291121188 2023-02-13 12:25:21,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., pid=85, masterSystemTime=1676291121165 2023-02-13 12:25:21,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:21,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:21,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 31a6bfc09681f3a11506a4775f610e4d, NAME => 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 12:25:21,190 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:21,190 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291121190"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291121190"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291121190"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291121190"}]},"ts":"1676291121190"} 2023-02-13 12:25:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,191 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,192 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:21,192 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:21,192 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-02-13 12:25:21,192 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293 in 176 msec 2023-02-13 12:25:21,192 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31a6bfc09681f3a11506a4775f610e4d columnFamilyName family1 2023-02-13 12:25:21,193 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(310): Store=31a6bfc09681f3a11506a4775f610e4d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN in 346 msec 2023-02-13 12:25:21,194 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,195 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2 2023-02-13 12:25:21,195 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2 2023-02-13 12:25:21,195 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31a6bfc09681f3a11506a4775f610e4d columnFamilyName family2 2023-02-13 12:25:21,196 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(310): Store=31a6bfc09681f3a11506a4775f610e4d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:21,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:21,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:21,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 31a6bfc09681f3a11506a4775f610e4d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66770995, jitterRate=-0.005034640431404114}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:21,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 31a6bfc09681f3a11506a4775f610e4d: Running coprocessor pre-open hook at 1676291121190Writing region info on filesystem at 1676291121190Initializing all the Stores at 1676291121191 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121191Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121191Cleaning up temporary data from old regions at 1676291121197 (+6 ms)Cleaning up detritus from prior splits at 1676291121198 (+1 ms)Running coprocessor post-open hooks at 1676291121200 (+2 ms)Region opened successfully at 1676291121200 2023-02-13 12:25:21,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., pid=84, masterSystemTime=1676291121165 2023-02-13 12:25:21,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:21,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:21,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:21,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5d424be18d07d0a984d49d2450dc4b7e, NAME => 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 12:25:21,203 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:21,203 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291121203"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291121203"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291121203"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291121203"}]},"ts":"1676291121203"} 2023-02-13 12:25:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,204 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,205 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:21,205 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:21,205 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-02-13 12:25:21,206 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293 in 190 msec 2023-02-13 12:25:21,206 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d424be18d07d0a984d49d2450dc4b7e columnFamilyName family1 2023-02-13 12:25:21,206 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN in 358 msec 2023-02-13 12:25:21,207 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(310): Store=5d424be18d07d0a984d49d2450dc4b7e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,207 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,208 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2 2023-02-13 12:25:21,208 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2 2023-02-13 12:25:21,208 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d424be18d07d0a984d49d2450dc4b7e columnFamilyName family2 2023-02-13 12:25:21,209 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(310): Store=5d424be18d07d0a984d49d2450dc4b7e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:21,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:21,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:21,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:21,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5d424be18d07d0a984d49d2450dc4b7e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60405778, jitterRate=-0.09988376498222351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:21,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5d424be18d07d0a984d49d2450dc4b7e: Running coprocessor pre-open hook at 1676291121203Writing region info on filesystem at 1676291121203Initializing all the Stores at 1676291121203Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121204 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291121204Cleaning up temporary data from old regions at 1676291121210 (+6 ms)Cleaning up detritus from prior splits at 1676291121210Running coprocessor post-open hooks at 1676291121213 (+3 ms)Region opened successfully at 1676291121213 2023-02-13 12:25:21,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., pid=83, masterSystemTime=1676291121165 2023-02-13 12:25:21,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:21,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:21,215 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:21,215 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291121215"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291121215"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291121215"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291121215"}]},"ts":"1676291121215"} 2023-02-13 12:25:21,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 12:25:21,217 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293 in 204 msec 2023-02-13 12:25:21,218 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 12:25:21,218 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN in 371 msec 2023-02-13 12:25:21,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:25:21,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291121219"}]},"ts":"1676291121219"} 2023-02-13 12:25:21,219 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 12:25:21,255 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:25:21,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2950 sec 2023-02-13 12:25:21,365 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 12:25:22,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 12:25:22,073 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-13 12:25:22,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:22,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:22,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:22,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:22,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 12:25:22,096 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 12:25:22,096 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:22,096 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 12:25:22,096 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 12:25:22,096 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:22,096 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:22,107 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:22,107 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:22,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:22,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:22,108 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:22,108 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:22,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:22,108 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:22,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:22,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:22,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 12:25:22,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 12:25:22,109 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 12:25:22,109 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:22,109 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 12:25:22,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:22,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:22,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:22,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:22,112 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:22,112 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 534c6c8744730b7c32dbd5f29d42ddff 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:22,112 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:22,112 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 31a6bfc09681f3a11506a4775f610e4d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:22,112 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:22,113 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:22,114 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 5d424be18d07d0a984d49d2450dc4b7e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:22,535 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd 2023-02-13 12:25:22,537 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb 2023-02-13 12:25:22,537 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 2023-02-13 12:25:22,943 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/c74929eb321a4d5290fe17811940cf75 2023-02-13 12:25:22,943 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/8c9f1917816640868a9631ceb1e18a41 2023-02-13 12:25:22,944 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/d5ab874ed0e148e594270a51b6000ac7 2023-02-13 12:25:22,965 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3 2023-02-13 12:25:22,965 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f 2023-02-13 12:25:23,368 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8 2023-02-13 12:25:23,370 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/fe09d17213cf43cd86656fc231ac6211 2023-02-13 12:25:23,370 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/ff51486288cf40ea9a1a39ac978b4141 2023-02-13 12:25:23,376 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b 2023-02-13 12:25:23,376 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/d5ab874ed0e148e594270a51b6000ac7 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7 2023-02-13 12:25:23,376 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/c74929eb321a4d5290fe17811940cf75 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75 2023-02-13 12:25:23,381 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/8c9f1917816640868a9631ceb1e18a41 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41 2023-02-13 12:25:23,381 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:23,381 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:23,382 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/ff51486288cf40ea9a1a39ac978b4141 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141 2023-02-13 12:25:23,382 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/fe09d17213cf43cd86656fc231ac6211 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211 2023-02-13 12:25:23,385 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:23,385 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b 2023-02-13 12:25:23,385 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:23,385 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:23,386 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 534c6c8744730b7c32dbd5f29d42ddff in 1274ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:23,386 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 31a6bfc09681f3a11506a4775f610e4d in 1274ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:23,387 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:23,387 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:23,387 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:23,387 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:23,387 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:23,387 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:23,389 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:23,390 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 5d424be18d07d0a984d49d2450dc4b7e in 1277ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:23,390 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:23,390 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:23,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:23,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:23,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,390 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 12:25:23,391 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 12:25:23,424 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,424 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:23,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:23,424 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:23,425 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:23,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 12:25:23,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e72354e[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:23,428 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 12:25:23,428 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,437 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,437 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 12:25:23,437 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 12:25:23,437 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:23,437 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,438 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:23,445 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 12:25:23,445 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:23,445 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,445 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 12:25:23,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:23,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:23,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:23,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:23,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:23,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:23,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 12:25:23,452 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 12:25:23,452 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:23,452 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:23,452 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 12:25:23,452 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:23,462 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,462 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,462 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:23,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:23,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:23,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:23,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:23,463 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:23,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:23,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,466 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:23,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,479 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,479 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:23,479 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:23,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:23,479 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:23,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:23,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 12:25:23,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:23,479 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:23,479 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:23,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 12:25:23,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:23,480 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:23,480 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,480 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:23,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:23,480 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,481 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:23,481 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,581 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:23,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:23,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:23,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:23,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:23,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:23,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:23,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 12:25:23,598 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 12:25:23,598 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:23,598 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 12:25:23,598 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 12:25:23,599 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,599 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:23,607 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:23,607 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:23,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:23,608 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:23,608 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:23,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:23,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:23,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:23,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:23,610 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 12:25:23,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 12:25:23,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 12:25:23,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 12:25:23,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:23,610 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:23,611 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 31a6bfc09681f3a11506a4775f610e4d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:23,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:23,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:23,611 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 534c6c8744730b7c32dbd5f29d42ddff 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:23,611 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:23,611 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 5d424be18d07d0a984d49d2450dc4b7e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:24,033 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc 2023-02-13 12:25:24,033 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 2023-02-13 12:25:24,034 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 2023-02-13 12:25:24,440 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/24cd311f92cf4563af2fdc457eb25dd9 2023-02-13 12:25:24,440 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/f9fcbc16aae24201884bd4618bb18dc8 2023-02-13 12:25:24,440 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/05f2875b409d4605b82237bc61edc387 2023-02-13 12:25:24,860 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0 2023-02-13 12:25:24,861 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b 2023-02-13 12:25:24,861 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d 2023-02-13 12:25:25,223 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:25:25,266 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/4caed0ca74854af3b5159470b6045d40 2023-02-13 12:25:25,269 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/dcbd6478a3bf4472900b7840ec522d32 2023-02-13 12:25:25,270 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/8df9237ff72543e5acce549ab471b4a3 2023-02-13 12:25:25,272 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/f9fcbc16aae24201884bd4618bb18dc8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8 2023-02-13 12:25:25,274 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/24cd311f92cf4563af2fdc457eb25dd9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9 2023-02-13 12:25:25,275 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/05f2875b409d4605b82237bc61edc387 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387 2023-02-13 12:25:25,278 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:25,278 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/4caed0ca74854af3b5159470b6045d40 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40 2023-02-13 12:25:25,279 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:25,279 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:25,280 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/8df9237ff72543e5acce549ab471b4a3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3 2023-02-13 12:25:25,280 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/dcbd6478a3bf4472900b7840ec522d32 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32 2023-02-13 12:25:25,283 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:25,284 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 534c6c8744730b7c32dbd5f29d42ddff in 1673ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:25,284 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:25,284 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:25,285 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:25,285 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:25,285 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:25,285 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 5d424be18d07d0a984d49d2450dc4b7e in 1674ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:25,286 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:25,286 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 31a6bfc09681f3a11506a4775f610e4d in 1675ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:25,286 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:25,286 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:25,286 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 12:25:25,286 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 12:25:25,329 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,329 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:25,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:25,330 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:25,330 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:25,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:25,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 12:25:25,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2564e8c0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:25,334 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 12:25:25,334 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,345 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 12:25:25,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 12:25:25,346 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:25,346 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,346 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:25,357 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 12:25:25,358 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:25,357 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,358 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 12:25:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:25,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:25,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:25,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:25,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:25,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 12:25:25,368 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 12:25:25,368 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:25,368 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:25,368 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 12:25:25,368 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:25,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:25,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:25,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:25,380 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,382 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:25,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,395 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,395 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:25,395 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:25,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:25,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 12:25:25,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:25,395 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:25,395 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:25,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 12:25:25,395 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:25,396 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,396 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,396 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:25,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:25,396 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:25,396 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:25,397 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,497 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:25,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:25,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:25,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:25,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:25,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:25,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:25,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 12:25:25,511 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 12:25:25,511 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:25,511 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 12:25:25,511 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 12:25:25,512 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,512 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:25,520 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:25,520 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:25,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:25,521 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:25,521 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:25,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:25,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:25,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:25,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:25,522 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 12:25:25,522 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 12:25:25,522 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 12:25:25,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 12:25:25,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:25,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:25,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:25,522 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:25,522 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:25,522 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 31a6bfc09681f3a11506a4775f610e4d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:25,523 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 5d424be18d07d0a984d49d2450dc4b7e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:25,522 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 534c6c8744730b7c32dbd5f29d42ddff 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:25,941 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 2023-02-13 12:25:25,941 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd 2023-02-13 12:25:25,941 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af 2023-02-13 12:25:26,346 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/703aa149c34d4fcfbf96ea9de8422de0 2023-02-13 12:25:26,347 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/f63b749fb7484be5b9fd853163640e64 2023-02-13 12:25:26,347 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/8b93ecba1939429bbeea19426a3e363e 2023-02-13 12:25:26,782 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427 2023-02-13 12:25:26,782 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881 2023-02-13 12:25:26,782 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2 2023-02-13 12:25:26,793 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/64c4490452fd49b9be8e50b18403c2fc 2023-02-13 12:25:26,797 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/f63b749fb7484be5b9fd853163640e64 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64 2023-02-13 12:25:26,800 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:26,801 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/64c4490452fd49b9be8e50b18403c2fc as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc 2023-02-13 12:25:26,804 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:26,805 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 5d424be18d07d0a984d49d2450dc4b7e in 1283ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:26,805 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:26,805 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:26,805 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:27,191 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/c04b7ac97e83473093f786135c648cb7 2023-02-13 12:25:27,210 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/67a2ab442e3f4c0d838f9712651614c6 2023-02-13 12:25:27,214 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/8b93ecba1939429bbeea19426a3e363e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e 2023-02-13 12:25:27,219 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/703aa149c34d4fcfbf96ea9de8422de0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0 2023-02-13 12:25:27,221 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:27,223 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/c04b7ac97e83473093f786135c648cb7 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7 2023-02-13 12:25:27,225 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:27,226 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/67a2ab442e3f4c0d838f9712651614c6 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6 2023-02-13 12:25:27,228 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:27,229 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 31a6bfc09681f3a11506a4775f610e4d in 1707ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:27,229 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:27,229 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:27,229 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:27,231 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:27,232 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 534c6c8744730b7c32dbd5f29d42ddff in 1710ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:27,232 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:27,232 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:27,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:27,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:27,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 12:25:27,232 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 12:25:27,245 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,245 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:27,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:27,245 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,245 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:27,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:27,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 12:25:27,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216c7f37[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:27,247 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 12:25:27,247 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,280 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,280 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 12:25:27,280 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 12:25:27,280 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,280 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:27,295 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 12:25:27,295 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,295 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:27,296 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 12:25:27,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:27,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:27,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:27,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:27,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:27,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:27,305 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 12:25:27,305 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 12:25:27,305 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:27,305 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:27,305 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 12:25:27,305 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:27,316 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,316 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,316 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:27,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:27,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:27,318 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:27,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,322 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:27,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 12:25:27,332 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:27,332 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:27,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:27,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:27,333 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:27,333 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:27,332 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-13 12:25:27,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:27,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:27,433 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:27,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:27,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:27,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:27,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:27,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:27,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:27,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 12:25:27,449 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 12:25:27,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:27,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 12:25:27,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 12:25:27,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,450 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:27,457 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,457 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:27,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:27,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:27,457 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:27,457 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 12:25:27,458 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-13 12:25:27,458 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 12:25:27,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 12:25:27,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:27,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:27,459 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 31a6bfc09681f3a11506a4775f610e4d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:27,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:27,459 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:27,459 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 534c6c8744730b7c32dbd5f29d42ddff 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:27,459 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:27,459 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 5d424be18d07d0a984d49d2450dc4b7e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:27,471 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc 2023-02-13 12:25:27,877 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/152389f341ff4965a4002bff15921154 2023-02-13 12:25:27,879 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f 2023-02-13 12:25:27,879 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 2023-02-13 12:25:28,286 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/fa3fca54a6c447e1ae69908c79d01f56 2023-02-13 12:25:28,286 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/e1a4c245b0574e5dbebf726279018d69 2023-02-13 12:25:28,297 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e 2023-02-13 12:25:28,704 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/7111cbc60e39458dbc3d5cda1ab87273 2023-02-13 12:25:28,713 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84 2023-02-13 12:25:28,714 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b 2023-02-13 12:25:28,714 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/152389f341ff4965a4002bff15921154 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154 2023-02-13 12:25:28,719 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:28,720 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/7111cbc60e39458dbc3d5cda1ab87273 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273 2023-02-13 12:25:28,724 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:28,725 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 5d424be18d07d0a984d49d2450dc4b7e in 1266ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:28,725 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:28,725 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:28,725 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:29,121 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/948bf581acc3484aac27dd70302439ed 2023-02-13 12:25:29,121 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/004f2c9b6a124dbaa44601754a2a4761 2023-02-13 12:25:29,132 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/e1a4c245b0574e5dbebf726279018d69 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69 2023-02-13 12:25:29,132 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/fa3fca54a6c447e1ae69908c79d01f56 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56 2023-02-13 12:25:29,137 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:29,137 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:29,138 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/948bf581acc3484aac27dd70302439ed as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed 2023-02-13 12:25:29,138 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/004f2c9b6a124dbaa44601754a2a4761 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761 2023-02-13 12:25:29,143 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:29,143 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:29,144 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 31a6bfc09681f3a11506a4775f610e4d in 1685ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:29,144 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 534c6c8744730b7c32dbd5f29d42ddff in 1685ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:29,144 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:29,144 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:29,144 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:29,144 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 12:25:29,144 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 12:25:29,178 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,178 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:29,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:29,179 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,179 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:29,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:29,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:29,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:29,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 12:25:29,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f3b7e5c[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:29,180 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 12:25:29,180 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,190 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,190 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 12:25:29,190 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 12:25:29,190 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:29,191 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,191 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:29,199 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 12:25:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:29,199 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 12:25:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:29,203 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 12:25:29,203 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:29,203 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:29,203 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 12:25:29,203 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:29,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 12:25:29,212 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,212 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:29,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,212 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:29,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:29,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:29,213 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,215 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:29,224 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:29,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:29,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:29,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 12:25:29,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:29,224 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:29,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:29,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:29,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:29,225 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:29,225 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:29,326 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:29,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:29,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1580): Client=jenkins//188.40.62.62 snapshot request for:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:29,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676291129345 (current time:1676291129345). 2023-02-13 12:25:29,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 12:25:29,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] zookeeper.ReadOnlyZKClient(139): Connect 0x6354ed51 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:29,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11788290, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:29,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:29,366 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:60088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:29,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6354ed51 to 127.0.0.1:64508 2023-02-13 12:25:29,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:29,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 12:25:29,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:29,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:29,380 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:29,381 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676291729381 2023-02-13 12:25:29,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:29,381 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1676291129328 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 12:25:29,383 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:29,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 12:25:29,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-02-13 12:25:29,385 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 12:25:29,386 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1676291729386 2023-02-13 12:25:29,388 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:29,389 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 12:25:29,489 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:29,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:29,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:29,496 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 12:25:29,697 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:29,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:29,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:29,700 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 12:25:29,800 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1676291129328 2023-02-13 12:25:29,801 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1676291129328' 2023-02-13 12:25:29,801 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 12:25:29,801 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1676291129328' starting 'acquire' 2023-02-13 12:25:29,801 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1676291129328', kicking off acquire phase on members. 2023-02-13 12:25:29,803 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:29,803 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1676291129328 2023-02-13 12:25:29,837 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 12:25:29,837 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 12:25:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:25:29,838 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,838 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1676291129328 2023-02-13 12:25:29,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-13 12:25:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1676291129328 2023-02-13 12:25:29,841 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1676291129328 from table testCompactionWithHFileLink type FLUSH 2023-02-13 12:25:29,846 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1676291129328 2023-02-13 12:25:29,846 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1676291129328' with timeout 300000ms 2023-02-13 12:25:29,846 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-13 12:25:29,847 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1676291129328' starting 'acquire' stage 2023-02-13 12:25:29,847 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1676291129328' locally acquired 2023-02-13 12:25:29,847 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (snaptb-1676291129328) in zk 2023-02-13 12:25:29,857 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,857 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 12:25:29,857 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,857 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1676291129328' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:29,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:29,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'snaptb-1676291129328' on coordinator 2023-02-13 12:25:29,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20dcf0b0[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:29,860 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1676291129328' starting 'in-barrier' execution. 2023-02-13 12:25:29,860 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,870 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:29,870 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1676291129328' received 'reached' from coordinator. 2023-02-13 12:25:29,871 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:29,871 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:29,872 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:29,872 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:29,872 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-13 12:25:29,872 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:29,872 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-13 12:25:29,872 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:29,873 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:29,872 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:29,873 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:29,873 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:29,873 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:29,874 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.' region-info for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,874 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.' region-info for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,874 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.' region-info for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,882 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 12:25:29,882 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 12:25:29,882 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-13 12:25:29,885 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69] hfiles 2023-02-13 12:25:29,885 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56] hfiles 2023-02-13 12:25:29,885 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,885 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154] hfiles 2023-02-13 12:25:29,886 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,885 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed] hfiles 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273] hfiles 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761] hfiles 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,889 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7 for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed for snapshot=snaptb-1676291129328 2023-02-13 12:25:29,890 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761 for snapshot=snaptb-1676291129328 2023-02-13 12:25:30,000 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:30,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:30,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:30,002 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 12:25:30,296 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. completed. 2023-02-13 12:25:30,296 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:30,296 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. completed. 2023-02-13 12:25:30,296 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:30,297 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. completed. 2023-02-13 12:25:30,297 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:30,297 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1676291129328' locally completed 2023-02-13 12:25:30,298 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1676291129328' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:30,341 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1676291129328' has notified controller of completion 2023-02-13 12:25:30,341 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:30,341 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,341 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1676291129328' completed. 2023-02-13 12:25:30,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:30,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 12:25:30,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:30,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:30,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:30,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:30,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:30,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1676291129328' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:30,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'snaptb-1676291129328', counting down latch. Waiting for 0 more 2023-02-13 12:25:30,351 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1676291129328' execution completed 2023-02-13 12:25:30,351 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:30,352 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:30,352 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1676291129328 2023-02-13 12:25:30,352 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1676291129328including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-13 12:25:30,362 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,362 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 12:25:30,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,362 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-13 12:25:30,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 12:25:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:25:30,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:30,364 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:30,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:30,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676291129328 2023-02-13 12:25:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-13 12:25:30,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676291129328 2023-02-13 12:25:30,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-13 12:25:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 12:25:30,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676291129328 2023-02-13 12:25:30,378 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-13 12:25:30,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676291129328/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:30,379 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1676291129328 2023-02-13 12:25:30,379 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:30,379 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-13 12:25:30,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-13 12:25:30,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 12:25:30,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:30,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676291129328 2023-02-13 12:25:30,379 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1676291129328 2023-02-13 12:25:30,379 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676291129328 2023-02-13 12:25:30,379 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-13 12:25:30,381 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af] mob files 2023-02-13 12:25:30,381 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb 2023-02-13 12:25:30,381 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd 2023-02-13 12:25:30,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af 2023-02-13 12:25:30,383 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e] mob files 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8 2023-02-13 12:25:30,384 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e 2023-02-13 12:25:30,503 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:30,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:30,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:30,504 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 12:25:30,792 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1676291129328 2023-02-13 12:25:30,919 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:25:30,919 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x352cbd47 to 127.0.0.1:64508 2023-02-13 12:25:30,919 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:30,919 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-13 12:25:30,920 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/snaptb-1676291129328 2023-02-13 12:25:31,365 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/snaptb-1676291129328 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/snaptb-1676291129328 2023-02-13 12:25:31,505 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:31,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:31,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:31,507 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-13 12:25:32,188 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1676291129328 of table testCompactionWithHFileLink completed 2023-02-13 12:25:32,188 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/snaptb-1676291129328 2023-02-13 12:25:32,190 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/snaptb-1676291129328 2023-02-13 12:25:32,190 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1676291129328 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676291129382Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1676291130791 (+1409 ms)Consolidate snapshot: snaptb-1676291129328 at 1676291130792 (+1 ms)Loading Region manifests for snaptb-1676291129328 at 1676291130792Writing data manifest for snaptb-1676291129328 at 1676291130932 (+140 ms)Verifying snapshot: snaptb-1676291129328 at 1676291131339 (+407 ms)Snapshot snaptb-1676291129328 of table testCompactionWithHFileLink completed at 1676291132188 (+849 ms) 2023-02-13 12:25:32,193 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-13 12:25:32,196 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.8100 sec 2023-02-13 12:25:33,507 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:33,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:33,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 12:25:33,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:33,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:33,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-13 12:25:33,558 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-13 12:25:33,559 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:33,559 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-13 12:25:33,559 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-13 12:25:33,559 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:33,559 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:33,595 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:33,595 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:33,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:33,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:33,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:33,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:33,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:33,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:33,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:33,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:33,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-13 12:25:33,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-13 12:25:33,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-13 12:25:33,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:33,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-13 12:25:33,599 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:33,599 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:33,599 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. started... 2023-02-13 12:25:33,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:33,600 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:33,600 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 31a6bfc09681f3a11506a4775f610e4d 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:33,600 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:33,600 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. started... 2023-02-13 12:25:33,600 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. started... 2023-02-13 12:25:33,601 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 534c6c8744730b7c32dbd5f29d42ddff 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:33,601 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 5d424be18d07d0a984d49d2450dc4b7e 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:34,423 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/a4185ec8f12c4fed9ae23b479674263c 2023-02-13 12:25:34,424 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/ffdc75fa9e3e4db78becfd7a29d74258 2023-02-13 12:25:34,424 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/1bdb44eb002946dd878dfccadfb76595 2023-02-13 12:25:34,432 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a4185ec8f12c4fed9ae23b479674263c 2023-02-13 12:25:34,433 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffdc75fa9e3e4db78becfd7a29d74258 2023-02-13 12:25:34,433 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1bdb44eb002946dd878dfccadfb76595 2023-02-13 12:25:34,854 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/d3bb44d483a947aaabefd4172fc6cb96 2023-02-13 12:25:34,859 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bb44d483a947aaabefd4172fc6cb96 2023-02-13 12:25:34,859 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/a4185ec8f12c4fed9ae23b479674263c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c 2023-02-13 12:25:34,863 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a4185ec8f12c4fed9ae23b479674263c 2023-02-13 12:25:34,863 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:34,863 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/d3bb44d483a947aaabefd4172fc6cb96 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96 2023-02-13 12:25:34,867 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d3bb44d483a947aaabefd4172fc6cb96 2023-02-13 12:25:34,867 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:34,867 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 5d424be18d07d0a984d49d2450dc4b7e in 1266ms, sequenceid=24, compaction requested=true 2023-02-13 12:25:34,867 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:34,867 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:34,868 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:35,257 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/dff91f6345494e5389eeca121272c3aa 2023-02-13 12:25:35,257 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/a6768661690a490a959488c08239c125 2023-02-13 12:25:35,262 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dff91f6345494e5389eeca121272c3aa 2023-02-13 12:25:35,262 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6768661690a490a959488c08239c125 2023-02-13 12:25:35,263 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/ffdc75fa9e3e4db78becfd7a29d74258 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258 2023-02-13 12:25:35,263 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/1bdb44eb002946dd878dfccadfb76595 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595 2023-02-13 12:25:35,267 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1bdb44eb002946dd878dfccadfb76595 2023-02-13 12:25:35,267 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:35,268 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ffdc75fa9e3e4db78becfd7a29d74258 2023-02-13 12:25:35,268 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258, entries=3, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:35,268 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/a6768661690a490a959488c08239c125 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125 2023-02-13 12:25:35,269 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/dff91f6345494e5389eeca121272c3aa as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa 2023-02-13 12:25:35,273 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a6768661690a490a959488c08239c125 2023-02-13 12:25:35,273 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dff91f6345494e5389eeca121272c3aa 2023-02-13 12:25:35,273 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:35,273 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa, entries=1, sequenceid=24, filesize=5.0 K 2023-02-13 12:25:35,274 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 534c6c8744730b7c32dbd5f29d42ddff in 1673ms, sequenceid=24, compaction requested=true 2023-02-13 12:25:35,274 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 31a6bfc09681f3a11506a4775f610e4d in 1674ms, sequenceid=24, compaction requested=true 2023-02-13 12:25:35,274 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:35,274 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:35,274 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:35,274 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:35,274 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:35,275 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:35,275 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:35,275 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,275 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-13 12:25:35,275 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-13 12:25:35,344 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,344 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:35,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:35,345 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,345 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:35,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:35,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:35,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:35,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-13 12:25:35,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22fc2e83[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:35,349 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-13 12:25:35,349 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,365 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,366 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-13 12:25:35,366 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-13 12:25:35,366 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:35,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,366 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:35,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-13 12:25:35,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:35,374 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,374 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-13 12:25:35,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:35,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:35,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:35,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-13 12:25:35,382 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-13 12:25:35,382 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:35,382 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:35,382 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-13 12:25:35,382 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:35,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:35,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,390 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:35,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:35,392 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:35,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:35,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,394 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:35,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-13 12:25:35,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:35,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:35,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-13 12:25:35,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-13 12:25:35,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:35,407 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:35,407 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:35,408 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:35,408 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:35,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:35,408 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-13 12:25:35,408 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-13 12:25:35,508 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:35,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:35,512 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:35,513 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 31a6bfc09681f3a11506a4775f610e4d/family1 is initiating major compaction (all files) 2023-02-13 12:25:35,513 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:35,514 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:35,514 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 31a6bfc09681f3a11506a4775f610e4d/family1 in testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:35,515 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp, totalSize=26.7 K 2023-02-13 12:25:35,516 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting d5ab874ed0e148e594270a51b6000ac7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122091 2023-02-13 12:25:35,517 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 24cd311f92cf4563af2fdc457eb25dd9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123587 2023-02-13 12:25:35,518 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 8b93ecba1939429bbeea19426a3e363e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125504 2023-02-13 12:25:35,519 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting fa3fca54a6c447e1ae69908c79d01f56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127440 2023-02-13 12:25:35,519 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 1bdb44eb002946dd878dfccadfb76595, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:35,532 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 31a6bfc09681f3a11506a4775f610e4d#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:35,946 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del 2023-02-13 12:25:35,946 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del 2023-02-13 12:25:36,768 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff3315b184914c5d970100bfaebd9c65 2023-02-13 12:25:36,769 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/ff3315b184914c5d970100bfaebd9c65 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/ff3315b184914c5d970100bfaebd9c65 2023-02-13 12:25:36,775 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff3315b184914c5d970100bfaebd9c65 2023-02-13 12:25:36,777 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 31a6bfc09681f3a11506a4775f610e4d/family1 of 31a6bfc09681f3a11506a4775f610e4d into ff3315b184914c5d970100bfaebd9c65(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:36,777 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:36,777 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:36,777 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 31a6bfc09681f3a11506a4775f610e4d/family2 is initiating major compaction (all files) 2023-02-13 12:25:36,778 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:36,778 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:36,778 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 31a6bfc09681f3a11506a4775f610e4d/family2 in testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:36,778 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp, totalSize=25.4 K 2023-02-13 12:25:36,779 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting fe09d17213cf43cd86656fc231ac6211, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122091 2023-02-13 12:25:36,779 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting dcbd6478a3bf4472900b7840ec522d32, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123587 2023-02-13 12:25:36,780 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting c04b7ac97e83473093f786135c648cb7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125504 2023-02-13 12:25:36,780 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 004f2c9b6a124dbaa44601754a2a4761, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127440 2023-02-13 12:25:36,781 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting a6768661690a490a959488c08239c125, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:36,797 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 31a6bfc09681f3a11506a4775f610e4d#family2#compaction#109 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:36,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595] to archive 2023-02-13 12:25:36,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:36,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/d5ab874ed0e148e594270a51b6000ac7 2023-02-13 12:25:36,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/24cd311f92cf4563af2fdc457eb25dd9 2023-02-13 12:25:36,828 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/8b93ecba1939429bbeea19426a3e363e 2023-02-13 12:25:36,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/fa3fca54a6c447e1ae69908c79d01f56 2023-02-13 12:25:36,832 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/1bdb44eb002946dd878dfccadfb76595 2023-02-13 12:25:37,214 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del 2023-02-13 12:25:37,214 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del 2023-02-13 12:25:38,047 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a37dff1ab27459f8029ed8e8e828de2 2023-02-13 12:25:38,048 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/6a37dff1ab27459f8029ed8e8e828de2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/6a37dff1ab27459f8029ed8e8e828de2 2023-02-13 12:25:38,052 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a37dff1ab27459f8029ed8e8e828de2 2023-02-13 12:25:38,053 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 31a6bfc09681f3a11506a4775f610e4d/family2 of 31a6bfc09681f3a11506a4775f610e4d into 6a37dff1ab27459f8029ed8e8e828de2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:38,053 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:38,053 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:38,053 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 534c6c8744730b7c32dbd5f29d42ddff/family1 is initiating major compaction (all files) 2023-02-13 12:25:38,053 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:38,053 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:38,053 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 534c6c8744730b7c32dbd5f29d42ddff/family1 in testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:38,053 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp, totalSize=26.7 K 2023-02-13 12:25:38,054 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting c74929eb321a4d5290fe17811940cf75, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122090 2023-02-13 12:25:38,054 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f9fcbc16aae24201884bd4618bb18dc8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:38,054 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 703aa149c34d4fcfbf96ea9de8422de0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125503 2023-02-13 12:25:38,054 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting e1a4c245b0574e5dbebf726279018d69, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127439 2023-02-13 12:25:38,055 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting ffdc75fa9e3e4db78becfd7a29d74258, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:38,065 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 534c6c8744730b7c32dbd5f29d42ddff#family1#compaction#110 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:38,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125] to archive 2023-02-13 12:25:38,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:38,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/fe09d17213cf43cd86656fc231ac6211 2023-02-13 12:25:38,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/dcbd6478a3bf4472900b7840ec522d32 2023-02-13 12:25:38,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/c04b7ac97e83473093f786135c648cb7 2023-02-13 12:25:38,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/004f2c9b6a124dbaa44601754a2a4761 2023-02-13 12:25:38,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/a6768661690a490a959488c08239c125 2023-02-13 12:25:38,475 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del 2023-02-13 12:25:38,475 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del 2023-02-13 12:25:39,307 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7be81f1bd434387a4aa0f7ba344fb4e 2023-02-13 12:25:39,308 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/d7be81f1bd434387a4aa0f7ba344fb4e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d7be81f1bd434387a4aa0f7ba344fb4e 2023-02-13 12:25:39,313 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7be81f1bd434387a4aa0f7ba344fb4e 2023-02-13 12:25:39,314 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 534c6c8744730b7c32dbd5f29d42ddff/family1 of 534c6c8744730b7c32dbd5f29d42ddff into d7be81f1bd434387a4aa0f7ba344fb4e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:39,314 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:39,314 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:39,315 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 534c6c8744730b7c32dbd5f29d42ddff/family2 is initiating major compaction (all files) 2023-02-13 12:25:39,315 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:39,315 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:39,315 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 534c6c8744730b7c32dbd5f29d42ddff/family2 in testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:39,315 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp, totalSize=25.4 K 2023-02-13 12:25:39,316 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting ff51486288cf40ea9a1a39ac978b4141, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122090 2023-02-13 12:25:39,316 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 4caed0ca74854af3b5159470b6045d40, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:39,317 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 67a2ab442e3f4c0d838f9712651614c6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125503 2023-02-13 12:25:39,317 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 948bf581acc3484aac27dd70302439ed, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127439 2023-02-13 12:25:39,317 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting dff91f6345494e5389eeca121272c3aa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:39,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258] to archive 2023-02-13 12:25:39,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:39,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/c74929eb321a4d5290fe17811940cf75 2023-02-13 12:25:39,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/f9fcbc16aae24201884bd4618bb18dc8 2023-02-13 12:25:39,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/703aa149c34d4fcfbf96ea9de8422de0 2023-02-13 12:25:39,326 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/e1a4c245b0574e5dbebf726279018d69 2023-02-13 12:25:39,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ffdc75fa9e3e4db78becfd7a29d74258 2023-02-13 12:25:39,331 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 534c6c8744730b7c32dbd5f29d42ddff#family2#compaction#111 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:39,752 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del 2023-02-13 12:25:39,753 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del 2023-02-13 12:25:40,669 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5da1db77be04886ba4887dee675f57e 2023-02-13 12:25:40,670 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/a5da1db77be04886ba4887dee675f57e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/a5da1db77be04886ba4887dee675f57e 2023-02-13 12:25:40,678 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5da1db77be04886ba4887dee675f57e 2023-02-13 12:25:40,680 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 534c6c8744730b7c32dbd5f29d42ddff/family2 of 534c6c8744730b7c32dbd5f29d42ddff into a5da1db77be04886ba4887dee675f57e(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:40,680 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:40,680 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:40,680 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 5d424be18d07d0a984d49d2450dc4b7e/family1 is initiating major compaction (all files) 2023-02-13 12:25:40,680 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:40,680 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:40,681 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 5d424be18d07d0a984d49d2450dc4b7e/family1 in testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:40,681 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp, totalSize=26.7 K 2023-02-13 12:25:40,681 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 8c9f1917816640868a9631ceb1e18a41, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122088 2023-02-13 12:25:40,682 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 05f2875b409d4605b82237bc61edc387, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:40,683 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f63b749fb7484be5b9fd853163640e64, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125502 2023-02-13 12:25:40,683 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 152389f341ff4965a4002bff15921154, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127438 2023-02-13 12:25:40,684 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting a4185ec8f12c4fed9ae23b479674263c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:40,697 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 5d424be18d07d0a984d49d2450dc4b7e#family1#compaction#112 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:40,705 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del 2023-02-13 12:25:40,705 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del 2023-02-13 12:25:40,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa] to archive 2023-02-13 12:25:40,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:40,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/ff51486288cf40ea9a1a39ac978b4141 2023-02-13 12:25:40,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/4caed0ca74854af3b5159470b6045d40 2023-02-13 12:25:40,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/67a2ab442e3f4c0d838f9712651614c6 2023-02-13 12:25:40,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/948bf581acc3484aac27dd70302439ed 2023-02-13 12:25:40,726 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/dff91f6345494e5389eeca121272c3aa 2023-02-13 12:25:41,534 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88308d8803314756bb7557178bcc651e 2023-02-13 12:25:41,536 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/88308d8803314756bb7557178bcc651e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/88308d8803314756bb7557178bcc651e 2023-02-13 12:25:41,544 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 88308d8803314756bb7557178bcc651e 2023-02-13 12:25:41,545 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5d424be18d07d0a984d49d2450dc4b7e/family1 of 5d424be18d07d0a984d49d2450dc4b7e into 88308d8803314756bb7557178bcc651e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:41,545 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:41,545 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:25:41,546 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 5d424be18d07d0a984d49d2450dc4b7e/family2 is initiating major compaction (all files) 2023-02-13 12:25:41,546 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:25:41,546 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:25:41,546 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 5d424be18d07d0a984d49d2450dc4b7e/family2 in testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:41,546 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp, totalSize=25.4 K 2023-02-13 12:25:41,547 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 5c77e63ead1f4c82a7ae40fbff1dbd9b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122088 2023-02-13 12:25:41,547 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 8df9237ff72543e5acce549ab471b4a3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:41,548 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 64c4490452fd49b9be8e50b18403c2fc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125502 2023-02-13 12:25:41,548 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 7111cbc60e39458dbc3d5cda1ab87273, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127438 2023-02-13 12:25:41,549 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting d3bb44d483a947aaabefd4172fc6cb96, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-13 12:25:41,560 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 5d424be18d07d0a984d49d2450dc4b7e#family2#compaction#113 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:25:41,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c] to archive 2023-02-13 12:25:41,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:41,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/8c9f1917816640868a9631ceb1e18a41 2023-02-13 12:25:41,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/05f2875b409d4605b82237bc61edc387 2023-02-13 12:25:41,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/f63b749fb7484be5b9fd853163640e64 2023-02-13 12:25:41,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/152389f341ff4965a4002bff15921154 2023-02-13 12:25:41,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/a4185ec8f12c4fed9ae23b479674263c 2023-02-13 12:25:41,977 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del 2023-02-13 12:25:41,977 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del 2023-02-13 12:25:42,806 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f43ca3cc68145708cd11abf6fb9f7a2 2023-02-13 12:25:42,807 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/5f43ca3cc68145708cd11abf6fb9f7a2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5f43ca3cc68145708cd11abf6fb9f7a2 2023-02-13 12:25:42,813 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f43ca3cc68145708cd11abf6fb9f7a2 2023-02-13 12:25:42,814 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5d424be18d07d0a984d49d2450dc4b7e/family2 of 5d424be18d07d0a984d49d2450dc4b7e into 5f43ca3cc68145708cd11abf6fb9f7a2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:42,814 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:42,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96] to archive 2023-02-13 12:25:42,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:42,821 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:42,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5c77e63ead1f4c82a7ae40fbff1dbd9b 2023-02-13 12:25:42,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/8df9237ff72543e5acce549ab471b4a3 2023-02-13 12:25:42,826 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:42,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/64c4490452fd49b9be8e50b18403c2fc 2023-02-13 12:25:42,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/7111cbc60e39458dbc3d5cda1ab87273 2023-02-13 12:25:42,832 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/d3bb44d483a947aaabefd4172fc6cb96 2023-02-13 12:25:42,835 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:42,836 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:42,844 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:25:42,878 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 12:25:42,881 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del 2023-02-13 12:25:42,884 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del 2023-02-13 12:25:42,887 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del 2023-02-13 12:25:42,888 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:42,888 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(139): Connect 0x32e26b9f to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:42,929 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46a186ba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:42,929 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:42,929 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:42,929 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:42,972 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5 2023-02-13 12:25:42,973 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:42,986 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:42,989 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:59760, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:42,996 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:25:42,998 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:56354, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:25:43,006 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/dd2baf139b9d45ce84bc54663d04af30 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 12:25:43,006 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/dd2baf139b9d45ce84bc54663d04af30}] 2023-02-13 12:25:43,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/dd2baf139b9d45ce84bc54663d04af30 for inclusion in 5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:43,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 12:25:43,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 12:25:43,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:43,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/dd2baf139b9d45ce84bc54663d04af30 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__ieia9k1agvuemudhc35n0sva29p303ivjjivbpi3jlclaato4lqc7mjoronskrkr/family1/dd2baf139b9d45ce84bc54663d04af30 2023-02-13 12:25:43,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__ieia9k1agvuemudhc35n0sva29p303ivjjivbpi3jlclaato4lqc7mjoronskrkr/family1/dd2baf139b9d45ce84bc54663d04af30 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ 2023-02-13 12:25:43,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__ieia9k1agvuemudhc35n0sva29p303ivjjivbpi3jlclaato4lqc7mjoronskrkr/family1/dd2baf139b9d45ce84bc54663d04af30 into 5d424be18d07d0a984d49d2450dc4b7e/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ - updating store file list. 2023-02-13 12:25:43,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ into 5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:43,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__ieia9k1agvuemudhc35n0sva29p303ivjjivbpi3jlclaato4lqc7mjoronskrkr/family1/dd2baf139b9d45ce84bc54663d04af30 into 5d424be18d07d0a984d49d2450dc4b7e/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_) 2023-02-13 12:25:43,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__ieia9k1agvuemudhc35n0sva29p303ivjjivbpi3jlclaato4lqc7mjoronskrkr/family1/dd2baf139b9d45ce84bc54663d04af30 2023-02-13 12:25:43,026 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,026 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,028 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc 2023-02-13 12:25:43,029 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 2023-02-13 12:25:43,030 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd 2023-02-13 12:25:43,032 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af 2023-02-13 12:25:43,032 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:43,032 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 12:25:43,032 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:43,379 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17 2023-02-13 12:25:43,380 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,406 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/fc7c481c25194b3e83a446d8e8780db9 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 12:25:43,408 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/fc7c481c25194b3e83a446d8e8780db9}] 2023-02-13 12:25:43,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/fc7c481c25194b3e83a446d8e8780db9 for inclusion in 31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:43,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 12:25:43,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 12:25:43,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:43,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/fc7c481c25194b3e83a446d8e8780db9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__a2rki8drjpbvanvejrfoi5l5t95lgt0i3chdvjhfuranh71vc6776r86msksrf3s/family1/fc7c481c25194b3e83a446d8e8780db9 2023-02-13 12:25:43,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__a2rki8drjpbvanvejrfoi5l5t95lgt0i3chdvjhfuranh71vc6776r86msksrf3s/family1/fc7c481c25194b3e83a446d8e8780db9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ 2023-02-13 12:25:43,415 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__a2rki8drjpbvanvejrfoi5l5t95lgt0i3chdvjhfuranh71vc6776r86msksrf3s/family1/fc7c481c25194b3e83a446d8e8780db9 into 31a6bfc09681f3a11506a4775f610e4d/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ - updating store file list. 2023-02-13 12:25:43,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ into 31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:43,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__a2rki8drjpbvanvejrfoi5l5t95lgt0i3chdvjhfuranh71vc6776r86msksrf3s/family1/fc7c481c25194b3e83a446d8e8780db9 into 31a6bfc09681f3a11506a4775f610e4d/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_) 2023-02-13 12:25:43,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__a2rki8drjpbvanvejrfoi5l5t95lgt0i3chdvjhfuranh71vc6776r86msksrf3s/family1/fc7c481c25194b3e83a446d8e8780db9 2023-02-13 12:25:43,422 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,422 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,424 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd 2023-02-13 12:25:43,425 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f 2023-02-13 12:25:43,427 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 2023-02-13 12:25:43,428 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc 2023-02-13 12:25:43,428 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:43,428 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 12:25:43,429 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:43,782 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f 2023-02-13 12:25:43,783 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,806 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/9b515309b5be4f5b8ff8f92c49342690 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 12:25:43,807 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/9b515309b5be4f5b8ff8f92c49342690}] 2023-02-13 12:25:43,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/9b515309b5be4f5b8ff8f92c49342690 for inclusion in 534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:43,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 12:25:43,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 12:25:43,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:43,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/9b515309b5be4f5b8ff8f92c49342690 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__b2qnbbc4404up470r1gb8lvvg3cvr57ahterh2sa0aeg92o01hpian6fvms09n4e/family1/9b515309b5be4f5b8ff8f92c49342690 2023-02-13 12:25:43,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__b2qnbbc4404up470r1gb8lvvg3cvr57ahterh2sa0aeg92o01hpian6fvms09n4e/family1/9b515309b5be4f5b8ff8f92c49342690 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ 2023-02-13 12:25:43,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__b2qnbbc4404up470r1gb8lvvg3cvr57ahterh2sa0aeg92o01hpian6fvms09n4e/family1/9b515309b5be4f5b8ff8f92c49342690 into 534c6c8744730b7c32dbd5f29d42ddff/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ - updating store file list. 2023-02-13 12:25:43,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ into 534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:43,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__b2qnbbc4404up470r1gb8lvvg3cvr57ahterh2sa0aeg92o01hpian6fvms09n4e/family1/9b515309b5be4f5b8ff8f92c49342690 into 534c6c8744730b7c32dbd5f29d42ddff/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_) 2023-02-13 12:25:43,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__b2qnbbc4404up470r1gb8lvvg3cvr57ahterh2sa0aeg92o01hpian6fvms09n4e/family1/9b515309b5be4f5b8ff8f92c49342690 2023-02-13 12:25:43,824 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,825 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,826 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb 2023-02-13 12:25:43,828 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 2023-02-13 12:25:43,829 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 2023-02-13 12:25:43,830 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 2023-02-13 12:25:43,830 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:43,831 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 12:25:43,831 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:43,831 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:43,831 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:43,832 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del] 2023-02-13 12:25:43,832 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,834 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302134cd7d605c5b14dc1964ca5ccf8cf5797_del 2023-02-13 12:25:43,834 INFO [Listener at localhost.localdomain/33629] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 12:25:43,834 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del] 2023-02-13 12:25:43,835 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,837 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021351ff5339b6ff4cf3ae1c5a3b55012749_del 2023-02-13 12:25:43,837 INFO [Listener at localhost.localdomain/33629] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 12:25:43,837 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del] 2023-02-13 12:25:43,837 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:43,839 DEBUG [Listener at localhost.localdomain/33629] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213155f26e3ad604fd7975800555bc4663a_del 2023-02-13 12:25:43,839 INFO [Listener at localhost.localdomain/33629] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-13 12:25:43,843 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:43,847 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:43,850 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:43,851 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:43,855 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-13 12:25:43,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testCompactionWithHFileLink 2023-02-13 12:25:43,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-13 12:25:43,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:25:43,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291143858"}]},"ts":"1676291143858"} 2023-02-13 12:25:43,859 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-13 12:25:43,894 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-13 12:25:43,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, UNASSIGN}] 2023-02-13 12:25:43,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, UNASSIGN 2023-02-13 12:25:43,897 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, UNASSIGN 2023-02-13 12:25:43,897 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, UNASSIGN 2023-02-13 12:25:43,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:43,898 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:43,898 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:43,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291143898"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291143898"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291143898"}]},"ts":"1676291143898"} 2023-02-13 12:25:43,898 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291143898"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291143898"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291143898"}]},"ts":"1676291143898"} 2023-02-13 12:25:43,898 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291143898"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291143898"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291143898"}]},"ts":"1676291143898"} 2023-02-13 12:25:43,900 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=89, state=RUNNABLE; CloseRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:43,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:43,901 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:43,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:25:44,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:44,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5d424be18d07d0a984d49d2450dc4b7e, disabling compactions & flushes 2023-02-13 12:25:44,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:44,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:44,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. after waiting 0 ms 2023-02-13 12:25:44,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:44,066 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 12:25:44,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:44,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5d424be18d07d0a984d49d2450dc4b7e: Waiting for close lock at 1676291144053Running coprocessor pre-close hooks at 1676291144053Disabling compacts and flushes for region at 1676291144053Disabling writes for close at 1676291144054 (+1 ms)Writing region close event to WAL at 1676291144057 (+3 ms)Running coprocessor post-close hooks at 1676291144067 (+10 ms)Closed at 1676291144068 (+1 ms) 2023-02-13 12:25:44,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:44,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:44,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 31a6bfc09681f3a11506a4775f610e4d, disabling compactions & flushes 2023-02-13 12:25:44,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:44,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:44,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. after waiting 0 ms 2023-02-13 12:25:44,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:44,071 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=CLOSED 2023-02-13 12:25:44,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291144071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291144071"}]},"ts":"1676291144071"} 2023-02-13 12:25:44,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=89 2023-02-13 12:25:44,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=89, state=SUCCESS; CloseRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293 in 173 msec 2023-02-13 12:25:44,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 12:25:44,077 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, UNASSIGN in 181 msec 2023-02-13 12:25:44,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:44,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 31a6bfc09681f3a11506a4775f610e4d: Waiting for close lock at 1676291144071Running coprocessor pre-close hooks at 1676291144071Disabling compacts and flushes for region at 1676291144071Disabling writes for close at 1676291144071Writing region close event to WAL at 1676291144073 (+2 ms)Running coprocessor post-close hooks at 1676291144077 (+4 ms)Closed at 1676291144078 (+1 ms) 2023-02-13 12:25:44,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:44,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:44,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 534c6c8744730b7c32dbd5f29d42ddff, disabling compactions & flushes 2023-02-13 12:25:44,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:44,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:44,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. after waiting 0 ms 2023-02-13 12:25:44,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:44,080 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=CLOSED 2023-02-13 12:25:44,080 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291144080"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291144080"}]},"ts":"1676291144080"} 2023-02-13 12:25:44,083 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-02-13 12:25:44,083 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293 in 180 msec 2023-02-13 12:25:44,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-13 12:25:44,084 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, UNASSIGN in 188 msec 2023-02-13 12:25:44,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:44,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 534c6c8744730b7c32dbd5f29d42ddff: Waiting for close lock at 1676291144079Running coprocessor pre-close hooks at 1676291144079Disabling compacts and flushes for region at 1676291144079Disabling writes for close at 1676291144080 (+1 ms)Writing region close event to WAL at 1676291144081 (+1 ms)Running coprocessor post-close hooks at 1676291144084 (+3 ms)Closed at 1676291144085 (+1 ms) 2023-02-13 12:25:44,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:44,087 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=CLOSED 2023-02-13 12:25:44,087 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291144087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291144087"}]},"ts":"1676291144087"} 2023-02-13 12:25:44,091 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-02-13 12:25:44,091 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293 in 189 msec 2023-02-13 12:25:44,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-02-13 12:25:44,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, UNASSIGN in 196 msec 2023-02-13 12:25:44,093 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291144093"}]},"ts":"1676291144093"} 2023-02-13 12:25:44,095 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-13 12:25:44,103 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-13 12:25:44,105 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 248 msec 2023-02-13 12:25:44,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 12:25:44,162 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-13 12:25:44,176 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:44,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1580): Client=jenkins//188.40.62.62 snapshot request for:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:44,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676291144177 (current time:1676291144177). 2023-02-13 12:25:44,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-13 12:25:44,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] zookeeper.ReadOnlyZKClient(139): Connect 0x291d3c96 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:44,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65012073, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:44,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:44,195 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:59774, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:44,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x291d3c96 to 127.0.0.1:64508 2023-02-13 12:25:44,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:25:44,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-13 12:25:44,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:44,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:44,199 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:44,200 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676291744200 2023-02-13 12:25:44,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:44,200 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1676291129328-1676291144176 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-13 12:25:44,200 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:44,200 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-13 12:25:44,300 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:44,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:44,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:44,305 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-13 12:25:44,505 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:44,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:44,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:44,509 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-13 12:25:44,611 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 12:25:44,615 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d7be81f1bd434387a4aa0f7ba344fb4e] hfiles 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/88308d8803314756bb7557178bcc651e] hfiles 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d7be81f1bd434387a4aa0f7ba344fb4e 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/88308d8803314756bb7557178bcc651e 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/ff3315b184914c5d970100bfaebd9c65] hfiles 2023-02-13 12:25:44,618 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5] hfiles 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/ff3315b184914c5d970100bfaebd9c65 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/a5da1db77be04886ba4887dee675f57e] hfiles 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/a5da1db77be04886ba4887dee675f57e 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5f43ca3cc68145708cd11abf6fb9f7a2] hfiles 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5f43ca3cc68145708cd11abf6fb9f7a2 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/6a37dff1ab27459f8029ed8e8e828de2] hfiles 2023-02-13 12:25:44,619 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/6a37dff1ab27459f8029ed8e8e828de2 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del] hfiles 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133208fc4f88304a2a8cb591d426d2a3a2 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302133761b5f9093441e882ef364814508e84 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302138adf5e7b05ed4a87957c94e71267c6e3 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213f4b2b5f24af24a9bbec549e0a453cab0 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302137399e34e17eb4e419e02efa563eb883d 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bc6e0d63d142a0be71e8f1568a9427 2023-02-13 12:25:44,625 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213c616582eea044763a15c660f5ea7ba1b 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213fc000fff50d94f8590ab25016a31cb3f 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213320c842eedc6443ca8b9f78c87c2819b 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213aebca6d3cf0d4fd3a88117e6c44c7881 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213ccfb5e33b03140e28638d6cf238691a8 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213d48e2e68859c4f278ac60490d15df34e 2023-02-13 12:25:44,626 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del 2023-02-13 12:25:44,809 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:44,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:44,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:44,811 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-13 12:25:45,038 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-13 12:25:45,039 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:45,040 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:45,312 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:45,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:45,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-13 12:25:45,315 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-13 12:25:45,471 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676291129328-1676291144176 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:46,316 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-13 12:25:46,317 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1676291129328-1676291144176 of table testCompactionWithHFileLink completed 2023-02-13 12:25:46,317 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:46,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-13 12:25:46,318 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:46,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-13 12:25:46,318 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1676291129328-1676291144176 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676291144200Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676291129328-1676291144176 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1676291144611 (+411 ms)Consolidate snapshot: hbase-failsafe-snaptb-1676291129328-1676291144176 at 1676291145039 (+428 ms)Loading Region manifests for hbase-failsafe-snaptb-1676291129328-1676291144176 at 1676291145039Writing data manifest for hbase-failsafe-snaptb-1676291129328-1676291144176 at 1676291145048 (+9 ms)Verifying snapshot: hbase-failsafe-snaptb-1676291129328-1676291144176 at 1676291145457 (+409 ms)Snapshot hbase-failsafe-snaptb-1676291129328-1676291144176 of table testCompactionWithHFileLink completed at 1676291146317 (+860 ms) 2023-02-13 12:25:46,322 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-13 12:25:46,325 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.1240 sec 2023-02-13 12:25:46,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676291129328" table: "testCompactionWithHFileLink" creation_time: 1676291129345 type: FLUSH version: 2 ttl: 0 ) 2023-02-13 12:25:46,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1676291129328 as table=testCompactionWithHFileLink 2023-02-13 12:25:46,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:25:46,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:25:46,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:25:46,775 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 12:25:46,778 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-13 12:25:46,778 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-13 12:25:46,780 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-13 12:25:46,787 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1676291129328" table: "testCompactionWithHFileLink" creation_time: 1676291129345 type: FLUSH version: 2 ttl: 0 2023-02-13 12:25:46,788 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink 2023-02-13 12:25:46,795 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-13 12:25:46,801 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230213aa47763fd3744b5792c14f337dd2ff17 2023-02-13 12:25:46,803 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302130a636cb949444269ad1c8623c609e42f 2023-02-13 12:25:46,805 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213779de6c1e64d4e63bdecabe0ede8b2f5 2023-02-13 12:25:46,805 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,809 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,814 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,818 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,823 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,827 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,832 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,836 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,840 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,843 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,846 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,849 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af of snapshot=snaptb-1676291129328 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-13 12:25:46,853 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230213e3cda46e40af4848844881ef7503e8e8_del 2023-02-13 12:25:46,854 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230213c5ca9bfd63d74f8aabc7248f50ff1d96_del 2023-02-13 12:25:46,854 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230213540aa2d46ffa4de5832abceaa32f4d2e_del 2023-02-13 12:25:46,855 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:46,855 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:46,855 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:46,857 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/eb6e426597aa4912abae9c818b474b69_SeqId_29_ 2023-02-13 12:25:46,857 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d7be81f1bd434387a4aa0f7ba344fb4e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d7be81f1bd434387a4aa0f7ba344fb4e 2023-02-13 12:25:46,857 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/63a3a022e578465f8b624bad9663d76c_SeqId_29_ 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/ff3315b184914c5d970100bfaebd9c65 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/ff3315b184914c5d970100bfaebd9c65 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d5ab874ed0e148e594270a51b6000ac7 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/408b50f2afb7446cbb8b4d330ce7fe05_SeqId_29_ 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/88308d8803314756bb7557178bcc651e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/88308d8803314756bb7557178bcc651e 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c74929eb321a4d5290fe17811940cf75 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,858 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8c9f1917816640868a9631ceb1e18a41 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,861 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 05f2875b409d4605b82237bc61edc387 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,861 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f9fcbc16aae24201884bd4618bb18dc8 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,861 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 24cd311f92cf4563af2fdc457eb25dd9 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,864 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 703aa149c34d4fcfbf96ea9de8422de0 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,864 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f63b749fb7484be5b9fd853163640e64 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,864 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8b93ecba1939429bbeea19426a3e363e of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,871 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e1a4c245b0574e5dbebf726279018d69 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,871 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 152389f341ff4965a4002bff15921154 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,872 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fa3fca54a6c447e1ae69908c79d01f56 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/a5da1db77be04886ba4887dee675f57e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/a5da1db77be04886ba4887dee675f57e 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5f43ca3cc68145708cd11abf6fb9f7a2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/5f43ca3cc68145708cd11abf6fb9f7a2 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/6a37dff1ab27459f8029ed8e8e828de2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/6a37dff1ab27459f8029ed8e8e828de2 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fe09d17213cf43cd86656fc231ac6211 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ff51486288cf40ea9a1a39ac978b4141 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,876 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5c77e63ead1f4c82a7ae40fbff1dbd9b of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,881 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dcbd6478a3bf4472900b7840ec522d32 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,881 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4caed0ca74854af3b5159470b6045d40 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,881 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8df9237ff72543e5acce549ab471b4a3 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,884 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c04b7ac97e83473093f786135c648cb7 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,884 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 67a2ab442e3f4c0d838f9712651614c6 of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,884 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 64c4490452fd49b9be8e50b18403c2fc of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,887 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 004f2c9b6a124dbaa44601754a2a4761 of snapshot=snaptb-1676291129328 to region=31a6bfc09681f3a11506a4775f610e4d table=testCompactionWithHFileLink 2023-02-13 12:25:46,887 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7111cbc60e39458dbc3d5cda1ab87273 of snapshot=snaptb-1676291129328 to region=5d424be18d07d0a984d49d2450dc4b7e table=testCompactionWithHFileLink 2023-02-13 12:25:46,887 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 948bf581acc3484aac27dd70302439ed of snapshot=snaptb-1676291129328 to region=534c6c8744730b7c32dbd5f29d42ddff table=testCompactionWithHFileLink 2023-02-13 12:25:46,891 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1676291129328" table: "testCompactionWithHFileLink" creation_time: 1676291129345 type: FLUSH version: 2 ttl: 0 2023-02-13 12:25:46,892 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676291146892"}]},"ts":"9223372036854775807"} 2023-02-13 12:25:46,892 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676291146892"}]},"ts":"9223372036854775807"} 2023-02-13 12:25:46,892 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676291146892"}]},"ts":"9223372036854775807"} 2023-02-13 12:25:46,894 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 12:25:46,894 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 31a6bfc09681f3a11506a4775f610e4d, NAME => 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 534c6c8744730b7c32dbd5f29d42ddff, NAME => 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 5d424be18d07d0a984d49d2450dc4b7e, NAME => 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.', STARTKEY => '', ENDKEY => '1'}] 2023-02-13 12:25:46,894 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291146893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291146893"}]},"ts":"1676291146893"} 2023-02-13 12:25:46,894 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291146893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291146893"}]},"ts":"1676291146893"} 2023-02-13 12:25:46,894 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291146893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291146893"}]},"ts":"1676291146893"} 2023-02-13 12:25:46,896 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 12:25:46,896 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-13 12:25:46,896 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 31a6bfc09681f3a11506a4775f610e4d, NAME => 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 534c6c8744730b7c32dbd5f29d42ddff, NAME => 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 5d424be18d07d0a984d49d2450dc4b7e, NAME => 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.', STARTKEY => '', ENDKEY => '1'}] 2023-02-13 12:25:46,897 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1676291129328 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-13 12:25:46,898 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676291129328" table: "testCompactionWithHFileLink" creation_time: 1676291129345 type: FLUSH version: 2 ttl: 0 ) in 542 msec 2023-02-13 12:25:46,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 12:25:46,973 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-13 12:25:46,973 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:46,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(748): Client=jenkins//188.40.62.62 delete name: "hbase-failsafe-snaptb-1676291129328-1676291144176" 2023-02-13 12:25:46,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1676291129328-1676291144176 2023-02-13 12:25:46,987 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-13 12:25:46,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$10(2372): Client=jenkins//188.40.62.62 enable testCompactionWithHFileLink 2023-02-13 12:25:46,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-13 12:25:46,999 INFO [PEWorker-1] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-13 12:25:47,000 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291146999"}]},"ts":"1676291146999"} 2023-02-13 12:25:47,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 12:25:47,001 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-13 12:25:47,037 DEBUG [PEWorker-1] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-13 12:25:47,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN}] 2023-02-13 12:25:47,039 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN 2023-02-13 12:25:47,039 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN 2023-02-13 12:25:47,039 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN 2023-02-13 12:25:47,039 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 12:25:47,039 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 12:25:47,040 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-13 12:25:47,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 12:25:47,192 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,192 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,192 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,192 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291147192"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291147192"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291147192"}]},"ts":"1676291147192"} 2023-02-13 12:25:47,192 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291147192"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291147192"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291147192"}]},"ts":"1676291147192"} 2023-02-13 12:25:47,192 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291147191"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291147191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291147191"}]},"ts":"1676291147191"} 2023-02-13 12:25:47,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=98, state=RUNNABLE; OpenRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:47,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:47,195 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=100, state=RUNNABLE; OpenRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:47,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 12:25:47,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:47,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 31a6bfc09681f3a11506a4775f610e4d, NAME => 'testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 12:25:47,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:47,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,356 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,358 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:47,358 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:47,358 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31a6bfc09681f3a11506a4775f610e4d columnFamilyName family1 2023-02-13 12:25:47,386 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9 2023-02-13 12:25:47,392 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e 2023-02-13 12:25:47,398 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7 2023-02-13 12:25:47,404 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56 2023-02-13 12:25:47,404 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(310): Store=31a6bfc09681f3a11506a4775f610e4d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,404 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,405 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2 2023-02-13 12:25:47,405 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2 2023-02-13 12:25:47,406 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 31a6bfc09681f3a11506a4775f610e4d columnFamilyName family2 2023-02-13 12:25:47,413 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761 2023-02-13 12:25:47,419 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7 2023-02-13 12:25:47,424 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32 2023-02-13 12:25:47,430 DEBUG [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211 2023-02-13 12:25:47,430 INFO [StoreOpener-31a6bfc09681f3a11506a4775f610e4d-1] regionserver.HStore(310): Store=31a6bfc09681f3a11506a4775f610e4d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:47,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:25:47,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 31a6bfc09681f3a11506a4775f610e4d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71388708, jitterRate=0.06377464532852173}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:47,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 31a6bfc09681f3a11506a4775f610e4d: Running coprocessor pre-open hook at 1676291147353Writing region info on filesystem at 1676291147353Initializing all the Stores at 1676291147355 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147355Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147355Cleaning up temporary data from old regions at 1676291147433 (+78 ms)Cleaning up detritus from prior splits at 1676291147433Running coprocessor post-open hooks at 1676291147436 (+3 ms)Region opened successfully at 1676291147436 2023-02-13 12:25:47,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., pid=103, masterSystemTime=1676291147346 2023-02-13 12:25:47,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:47,437 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:47,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:47,437 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:47,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:47,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:47,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:47,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 534c6c8744730b7c32dbd5f29d42ddff, NAME => 'testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 12:25:47,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=31a6bfc09681f3a11506a4775f610e4d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,439 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291147439"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291147439"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291147439"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291147439"}]},"ts":"1676291147439"} 2023-02-13 12:25:47,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:47,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,440 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:47,441 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1912): 31a6bfc09681f3a11506a4775f610e4d/family1 is initiating minor compaction (all files) 2023-02-13 12:25:47,441 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 31a6bfc09681f3a11506a4775f610e4d/family1 in testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:47,441 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp, totalSize=21.6 K 2023-02-13 12:25:47,441 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:47,441 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1912): 31a6bfc09681f3a11506a4775f610e4d/family2 is initiating minor compaction (all files) 2023-02-13 12:25:47,441 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 31a6bfc09681f3a11506a4775f610e4d/family2 in testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:25:47,441 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp, totalSize=20.4 K 2023-02-13 12:25:47,442 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,442 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122091 2023-02-13 12:25:47,442 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122091 2023-02-13 12:25:47,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=100 2023-02-13 12:25:47,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=100, state=SUCCESS; OpenRegionProcedure 31a6bfc09681f3a11506a4775f610e4d, server=jenkins-hbase16.apache.org,38471,1676291011293 in 245 msec 2023-02-13 12:25:47,443 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123587 2023-02-13 12:25:47,443 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:47,443 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:47,443 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123587 2023-02-13 12:25:47,444 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 534c6c8744730b7c32dbd5f29d42ddff columnFamilyName family1 2023-02-13 12:25:47,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=31a6bfc09681f3a11506a4775f610e4d, ASSIGN in 406 msec 2023-02-13 12:25:47,444 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125504 2023-02-13 12:25:47,444 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125504 2023-02-13 12:25:47,445 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127440 2023-02-13 12:25:47,445 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127440 2023-02-13 12:25:47,463 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0 2023-02-13 12:25:47,472 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75 2023-02-13 12:25:47,481 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69 2023-02-13 12:25:47,486 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): 31a6bfc09681f3a11506a4775f610e4d#family2#compaction#114 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:47,486 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 31a6bfc09681f3a11506a4775f610e4d#family1#compaction#115 average throughput is 0.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:47,497 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8 2023-02-13 12:25:47,500 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(310): Store=534c6c8744730b7c32dbd5f29d42ddff/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,500 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,501 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2 2023-02-13 12:25:47,501 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2 2023-02-13 12:25:47,501 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 534c6c8744730b7c32dbd5f29d42ddff columnFamilyName family2 2023-02-13 12:25:47,507 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40 2023-02-13 12:25:47,511 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6 2023-02-13 12:25:47,516 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed 2023-02-13 12:25:47,521 DEBUG [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141 2023-02-13 12:25:47,521 INFO [StoreOpener-534c6c8744730b7c32dbd5f29d42ddff-1] regionserver.HStore(310): Store=534c6c8744730b7c32dbd5f29d42ddff/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:47,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:25:47,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 534c6c8744730b7c32dbd5f29d42ddff; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66491411, jitterRate=-0.009200766682624817}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:47,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 534c6c8744730b7c32dbd5f29d42ddff: Running coprocessor pre-open hook at 1676291147439Writing region info on filesystem at 1676291147439Initializing all the Stores at 1676291147441 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147441Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147442 (+1 ms)Cleaning up temporary data from old regions at 1676291147523 (+81 ms)Cleaning up detritus from prior splits at 1676291147523Running coprocessor post-open hooks at 1676291147525 (+2 ms)Region opened successfully at 1676291147525 2023-02-13 12:25:47,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., pid=102, masterSystemTime=1676291147346 2023-02-13 12:25:47,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:25:47,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 12:25:47,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:47,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:47,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:47,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5d424be18d07d0a984d49d2450dc4b7e, NAME => 'testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 12:25:47,528 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=534c6c8744730b7c32dbd5f29d42ddff, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,528 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291147528"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291147528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291147528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291147528"}]},"ts":"1676291147528"} 2023-02-13 12:25:47,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:47,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,529 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,530 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:47,530 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:47,530 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=99 2023-02-13 12:25:47,530 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure 534c6c8744730b7c32dbd5f29d42ddff, server=jenkins-hbase16.apache.org,38471,1676291011293 in 335 msec 2023-02-13 12:25:47,530 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d424be18d07d0a984d49d2450dc4b7e columnFamilyName family1 2023-02-13 12:25:47,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=534c6c8744730b7c32dbd5f29d42ddff, ASSIGN in 493 msec 2023-02-13 12:25:47,536 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387 2023-02-13 12:25:47,541 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154 2023-02-13 12:25:47,546 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41 2023-02-13 12:25:47,554 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64 2023-02-13 12:25:47,554 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(310): Store=5d424be18d07d0a984d49d2450dc4b7e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,554 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,555 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2 2023-02-13 12:25:47,555 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2 2023-02-13 12:25:47,555 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d424be18d07d0a984d49d2450dc4b7e columnFamilyName family2 2023-02-13 12:25:47,561 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b 2023-02-13 12:25:47,565 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc 2023-02-13 12:25:47,569 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273 2023-02-13 12:25:47,572 DEBUG [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3 2023-02-13 12:25:47,573 INFO [StoreOpener-5d424be18d07d0a984d49d2450dc4b7e-1] regionserver.HStore(310): Store=5d424be18d07d0a984d49d2450dc4b7e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:47,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:47,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:25:47,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5d424be18d07d0a984d49d2450dc4b7e; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61350023, jitterRate=-0.08581341803073883}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:47,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5d424be18d07d0a984d49d2450dc4b7e: Running coprocessor pre-open hook at 1676291147528Writing region info on filesystem at 1676291147528Initializing all the Stores at 1676291147529 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147529Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291147529Cleaning up temporary data from old regions at 1676291147575 (+46 ms)Cleaning up detritus from prior splits at 1676291147575Running coprocessor post-open hooks at 1676291147578 (+3 ms)Region opened successfully at 1676291147578 2023-02-13 12:25:47,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., pid=101, masterSystemTime=1676291147346 2023-02-13 12:25:47,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 12:25:47,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 12:25:47,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:47,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:47,581 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=5d424be18d07d0a984d49d2450dc4b7e, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:47,581 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676291147581"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291147581"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291147581"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291147581"}]},"ts":"1676291147581"} 2023-02-13 12:25:47,584 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=98 2023-02-13 12:25:47,584 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=98, state=SUCCESS; OpenRegionProcedure 5d424be18d07d0a984d49d2450dc4b7e, server=jenkins-hbase16.apache.org,38471,1676291011293 in 389 msec 2023-02-13 12:25:47,586 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=98, resume processing ppid=97 2023-02-13 12:25:47,586 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5d424be18d07d0a984d49d2450dc4b7e, ASSIGN in 547 msec 2023-02-13 12:25:47,586 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291147586"}]},"ts":"1676291147586"} 2023-02-13 12:25:47,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-13 12:25:47,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 12:25:47,619 INFO [PEWorker-1] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-13 12:25:47,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 629 msec 2023-02-13 12:25:48,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-13 12:25:48,107 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-13 12:25:48,130 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:48,139 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:48,148 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:48,149 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:48,156 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:25:48,168 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-13 12:25:48,169 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:48,169 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(139): Connect 0x6dbcc7d9 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:25:48,203 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@eb4d514, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:25:48,204 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:48,204 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:48,204 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:48,323 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family2/d84376adda294b3e959733e15674fe89 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/d84376adda294b3e959733e15674fe89 2023-02-13 12:25:48,327 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-13 12:25:48,328 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 31a6bfc09681f3a11506a4775f610e4d/family2 of 31a6bfc09681f3a11506a4775f610e4d into d84376adda294b3e959733e15674fe89(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:48,328 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:48,328 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., storeName=31a6bfc09681f3a11506a4775f610e4d/family2, priority=12, startTime=1676291147437; duration=0sec 2023-02-13 12:25:48,328 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-13 12:25:48,329 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:48,331 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:48,331 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1912): 534c6c8744730b7c32dbd5f29d42ddff/family1 is initiating minor compaction (all files) 2023-02-13 12:25:48,331 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 534c6c8744730b7c32dbd5f29d42ddff/family1 in testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:48,331 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp, totalSize=21.6 K 2023-02-13 12:25:48,331 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122090 2023-02-13 12:25:48,332 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:48,332 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125503 2023-02-13 12:25:48,333 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127439 2023-02-13 12:25:48,344 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): 534c6c8744730b7c32dbd5f29d42ddff#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:48,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761] to archive 2023-02-13 12:25:48,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:48,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fe09d17213cf43cd86656fc231ac6211 2023-02-13 12:25:48,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-dcbd6478a3bf4472900b7840ec522d32 2023-02-13 12:25:48,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-c04b7ac97e83473093f786135c648cb7 2023-02-13 12:25:48,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family2/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-004f2c9b6a124dbaa44601754a2a4761 2023-02-13 12:25:48,727 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/.tmp/family1/b47a9771c43a48598989a98d64082d66 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/b47a9771c43a48598989a98d64082d66 2023-02-13 12:25:48,737 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 31a6bfc09681f3a11506a4775f610e4d/family1 of 31a6bfc09681f3a11506a4775f610e4d into b47a9771c43a48598989a98d64082d66(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:48,737 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:48,737 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., storeName=31a6bfc09681f3a11506a4775f610e4d/family1, priority=12, startTime=1676291147436; duration=1sec 2023-02-13 12:25:48,737 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-13 12:25:48,738 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:48,740 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:48,741 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1912): 5d424be18d07d0a984d49d2450dc4b7e/family2 is initiating minor compaction (all files) 2023-02-13 12:25:48,741 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5d424be18d07d0a984d49d2450dc4b7e/family2 in testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:48,741 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp, totalSize=20.4 K 2023-02-13 12:25:48,742 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122088 2023-02-13 12:25:48,742 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:48,743 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125502 2023-02-13 12:25:48,744 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127438 2023-02-13 12:25:48,761 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5d424be18d07d0a984d49d2450dc4b7e#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:48,763 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family1/ab7b377294ce42c495a301d6b1f1d623 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/ab7b377294ce42c495a301d6b1f1d623 2023-02-13 12:25:48,767 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 534c6c8744730b7c32dbd5f29d42ddff/family1 of 534c6c8744730b7c32dbd5f29d42ddff into ab7b377294ce42c495a301d6b1f1d623(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:48,767 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:48,767 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., storeName=534c6c8744730b7c32dbd5f29d42ddff/family1, priority=12, startTime=1676291147526; duration=0sec 2023-02-13 12:25:48,767 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 12:25:48,767 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:48,769 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:48,769 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1912): 5d424be18d07d0a984d49d2450dc4b7e/family1 is initiating minor compaction (all files) 2023-02-13 12:25:48,769 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5d424be18d07d0a984d49d2450dc4b7e/family1 in testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:25:48,769 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp, totalSize=21.6 K 2023-02-13 12:25:48,769 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122088 2023-02-13 12:25:48,770 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:48,770 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125502 2023-02-13 12:25:48,771 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127438 2023-02-13 12:25:48,782 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] throttle.PressureAwareThroughputController(145): 5d424be18d07d0a984d49d2450dc4b7e#family1#compaction#118 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:48,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56] to archive 2023-02-13 12:25:48,828 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:48,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-d5ab874ed0e148e594270a51b6000ac7 2023-02-13 12:25:48,832 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-24cd311f92cf4563af2fdc457eb25dd9 2023-02-13 12:25:48,835 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-8b93ecba1939429bbeea19426a3e363e 2023-02-13 12:25:48,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/testCompactionWithHFileLink=31a6bfc09681f3a11506a4775f610e4d-fa3fca54a6c447e1ae69908c79d01f56 2023-02-13 12:25:48,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69] to archive 2023-02-13 12:25:48,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:48,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-c74929eb321a4d5290fe17811940cf75 2023-02-13 12:25:48,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-f9fcbc16aae24201884bd4618bb18dc8 2023-02-13 12:25:48,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-703aa149c34d4fcfbf96ea9de8422de0 2023-02-13 12:25:48,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-e1a4c245b0574e5dbebf726279018d69 2023-02-13 12:25:49,073 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213dc674bcd45af4cd5a4ed99a929dff8c7 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230213dc674bcd45af4cd5a4ed99a929dff8c7 2023-02-13 12:25:49,073 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213f64fa480ac5540d399728d6f298dbcd9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230213f64fa480ac5540d399728d6f298dbcd9 2023-02-13 12:25:49,074 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,074 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,074 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133ee1308cceaf4a17a7f78e60211df183 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302133ee1308cceaf4a17a7f78e60211df183 2023-02-13 12:25:49,080 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,086 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:25:49,090 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:59780, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:25:49,098 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:25:49,102 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:56356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:25:49,123 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/90211cf4a4094c4398d3342d5482669c first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-13 12:25:49,123 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/d4b827c6d9d347628943266c6d9a51ec first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-13 12:25:49,123 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f42edeae73f84cdcad4fb5f194486df8 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-13 12:25:49,124 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/90211cf4a4094c4398d3342d5482669c}] 2023-02-13 12:25:49,124 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=32 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/d4b827c6d9d347628943266c6d9a51ec}] 2023-02-13 12:25:49,125 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f42edeae73f84cdcad4fb5f194486df8}] 2023-02-13 12:25:49,134 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/90211cf4a4094c4398d3342d5482669c for inclusion in 31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:49,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f42edeae73f84cdcad4fb5f194486df8 for inclusion in 534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:49,134 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/d4b827c6d9d347628943266c6d9a51ec for inclusion in 5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 12:25:49,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 31a6bfc09681f3a11506a4775f610e4d: 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213/family1/d4b827c6d9d347628943266c6d9a51ec to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__hguo4ed5deb0dbdd31la04glmcuit47vdmnuhpnovrblqo5fq3jkc92p756l3gmv/family1/d4b827c6d9d347628943266c6d9a51ec 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213/family1/90211cf4a4094c4398d3342d5482669c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__h65ca8tgcjlsnm3ebal0dq7d0c1l6ulcirth0boteejcbh7guj3ireg4ifubsb4f/family1/90211cf4a4094c4398d3342d5482669c 2023-02-13 12:25:49,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213/family1/f42edeae73f84cdcad4fb5f194486df8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__g5s2it1jfg09k7jdu9rk7s4dd0n0su74hmomoek206t5eov1kkofo7dm5rib9u5g/family1/f42edeae73f84cdcad4fb5f194486df8 2023-02-13 12:25:49,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__hguo4ed5deb0dbdd31la04glmcuit47vdmnuhpnovrblqo5fq3jkc92p756l3gmv/family1/d4b827c6d9d347628943266c6d9a51ec as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/3475e7dfc21544229c5f2cbe18e6ea41_SeqId_34_ 2023-02-13 12:25:49,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__g5s2it1jfg09k7jdu9rk7s4dd0n0su74hmomoek206t5eov1kkofo7dm5rib9u5g/family1/f42edeae73f84cdcad4fb5f194486df8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d04e82d1d3234efdbf8070f167c7bb3c_SeqId_35_ 2023-02-13 12:25:49,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__h65ca8tgcjlsnm3ebal0dq7d0c1l6ulcirth0boteejcbh7guj3ireg4ifubsb4f/family1/90211cf4a4094c4398d3342d5482669c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/0a3244842c2a4654a3834d4e6d4a1cb1_SeqId_36_ 2023-02-13 12:25:49,140 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__hguo4ed5deb0dbdd31la04glmcuit47vdmnuhpnovrblqo5fq3jkc92p756l3gmv/family1/d4b827c6d9d347628943266c6d9a51ec into 5d424be18d07d0a984d49d2450dc4b7e/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/3475e7dfc21544229c5f2cbe18e6ea41_SeqId_34_ - updating store file list. 2023-02-13 12:25:49,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__g5s2it1jfg09k7jdu9rk7s4dd0n0su74hmomoek206t5eov1kkofo7dm5rib9u5g/family1/f42edeae73f84cdcad4fb5f194486df8 into 534c6c8744730b7c32dbd5f29d42ddff/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d04e82d1d3234efdbf8070f167c7bb3c_SeqId_35_ - updating store file list. 2023-02-13 12:25:49,140 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__h65ca8tgcjlsnm3ebal0dq7d0c1l6ulcirth0boteejcbh7guj3ireg4ifubsb4f/family1/90211cf4a4094c4398d3342d5482669c into 31a6bfc09681f3a11506a4775f610e4d/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/0a3244842c2a4654a3834d4e6d4a1cb1_SeqId_36_ - updating store file list. 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d04e82d1d3234efdbf8070f167c7bb3c_SeqId_35_ into 534c6c8744730b7c32dbd5f29d42ddff/family1 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/0a3244842c2a4654a3834d4e6d4a1cb1_SeqId_36_ into 31a6bfc09681f3a11506a4775f610e4d/family1 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/3475e7dfc21544229c5f2cbe18e6ea41_SeqId_34_ into 5d424be18d07d0a984d49d2450dc4b7e/family1 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__h65ca8tgcjlsnm3ebal0dq7d0c1l6ulcirth0boteejcbh7guj3ireg4ifubsb4f/family1/90211cf4a4094c4398d3342d5482669c into 31a6bfc09681f3a11506a4775f610e4d/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/family1/0a3244842c2a4654a3834d4e6d4a1cb1_SeqId_36_) 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__g5s2it1jfg09k7jdu9rk7s4dd0n0su74hmomoek206t5eov1kkofo7dm5rib9u5g/family1/f42edeae73f84cdcad4fb5f194486df8 into 534c6c8744730b7c32dbd5f29d42ddff/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family1/d04e82d1d3234efdbf8070f167c7bb3c_SeqId_35_) 2023-02-13 12:25:49,143 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__hguo4ed5deb0dbdd31la04glmcuit47vdmnuhpnovrblqo5fq3jkc92p756l3gmv/family1/d4b827c6d9d347628943266c6d9a51ec into 5d424be18d07d0a984d49d2450dc4b7e/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/3475e7dfc21544229c5f2cbe18e6ea41_SeqId_34_) 2023-02-13 12:25:49,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__h65ca8tgcjlsnm3ebal0dq7d0c1l6ulcirth0boteejcbh7guj3ireg4ifubsb4f/family1/90211cf4a4094c4398d3342d5482669c 2023-02-13 12:25:49,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__g5s2it1jfg09k7jdu9rk7s4dd0n0su74hmomoek206t5eov1kkofo7dm5rib9u5g/family1/f42edeae73f84cdcad4fb5f194486df8 2023-02-13 12:25:49,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testCompactionWithHFileLink__hguo4ed5deb0dbdd31la04glmcuit47vdmnuhpnovrblqo5fq3jkc92p756l3gmv/family1/d4b827c6d9d347628943266c6d9a51ec 2023-02-13 12:25:49,146 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,146 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,146 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,146 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:49,147 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:49,147 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:49,148 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302130b4524ce3fb2481fb5a5a5c942d103fb 2023-02-13 12:25:49,148 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023021364946f08e7874af49a0d42ad1869cebc 2023-02-13 12:25:49,148 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302130ec3aba5791644ce9f7e24051b3cd5bd 2023-02-13 12:25:49,149 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213176e6c8e632d45b3bd097ee657afcc28 2023-02-13 12:25:49,149 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302137071bdc46001493aaee2735844901182 2023-02-13 12:25:49,149 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021343f001899f6a4e3f90a71efc31327f1f 2023-02-13 12:25:49,149 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302136a93abdeb3f148539fce7797f5d97170 2023-02-13 12:25:49,150 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302135c3a9d600db1497eb9aa6ae7b17dfa80 2023-02-13 12:25:49,150 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302137811d54edb2441a18baeb446ef055cfd 2023-02-13 12:25:49,150 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230213db4506aac7c0482aa385a65b957041f0 2023-02-13 12:25:49,150 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:49,150 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 12:25:49,151 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230213708b1a58cf3d453dab8509caf0be34bc 2023-02-13 12:25:49,151 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:49,151 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:49,151 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230213e1ec07b9525142b48b1b2b9e2ba216af 2023-02-13 12:25:49,151 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 12:25:49,151 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-13 12:25:49,151 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 12:25:49,151 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:49,151 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testCompactionWithHFileLink' and column='family1' 2023-02-13 12:25:49,151 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-13 12:25:49,154 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:49,158 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:49,164 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:49,165 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:49,173 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:49,174 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:49,179 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family2/017a12200e824b6db607099e60baef82 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/017a12200e824b6db607099e60baef82 2023-02-13 12:25:49,185 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5d424be18d07d0a984d49d2450dc4b7e/family2 of 5d424be18d07d0a984d49d2450dc4b7e into 017a12200e824b6db607099e60baef82(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:25:49,186 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:49,186 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., storeName=5d424be18d07d0a984d49d2450dc4b7e/family2, priority=12, startTime=1676291147579; duration=0sec 2023-02-13 12:25:49,186 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 12:25:49,186 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 12:25:49,188 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-13 12:25:49,189 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1912): 534c6c8744730b7c32dbd5f29d42ddff/family2 is initiating minor compaction (all files) 2023-02-13 12:25:49,189 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 534c6c8744730b7c32dbd5f29d42ddff/family2 in testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:25:49,189 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp, totalSize=20.4 K 2023-02-13 12:25:49,189 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291122090 2023-02-13 12:25:49,190 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291123586 2023-02-13 12:25:49,190 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291125503 2023-02-13 12:25:49,191 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291127439 2023-02-13 12:25:49,191 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=276 (was 234) - Thread LEAK? -, OpenFileDescriptor=745 (was 589) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=48 (was 79), ProcessCount=167 (was 167), AvailableMemoryMB=5882 (was 5977) 2023-02-13 12:25:49,203 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=275, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=48, ProcessCount=167, AvailableMemoryMB=5881 2023-02-13 12:25:49,204 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): 534c6c8744730b7c32dbd5f29d42ddff#family2#compaction#119 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:25:49,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$15(2997): Client=jenkins//188.40.62.62 creating {NAME => 'ns'} 2023-02-13 12:25:49,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-13 12:25:49,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 12:25:49,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273] to archive 2023-02-13 12:25:49,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:49,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-5c77e63ead1f4c82a7ae40fbff1dbd9b 2023-02-13 12:25:49,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8df9237ff72543e5acce549ab471b4a3 2023-02-13 12:25:49,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-64c4490452fd49b9be8e50b18403c2fc 2023-02-13 12:25:49,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family2/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-7111cbc60e39458dbc3d5cda1ab87273 2023-02-13 12:25:49,244 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 12:25:49,256 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 45 msec 2023-02-13 12:25:49,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-13 12:25:49,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 12:25:49,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-13 12:25:49,325 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:25:49,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-13 12:25:49,326 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:25:49,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:49,328 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5 2023-02-13 12:25:49,328 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:49,328 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:49,328 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5 empty. 2023-02-13 12:25:49,328 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc empty. 2023-02-13 12:25:49,329 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0 empty. 2023-02-13 12:25:49,329 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5 2023-02-13 12:25:49,329 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:49,329 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:49,329 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-13 12:25:49,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:49,621 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:25:49,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:49,738 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-13 12:25:49,739 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4993132286c530710e08273860fde1c5, NAME => 'ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:49,739 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => c410f61e2e0cf5f28a4b5c4a521e8eb0, NAME => 'ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:49,740 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => be0d091bd879acd2c49cb81bc7c86ecc, NAME => 'ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:25:49,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:50,010 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/.tmp/family1/4ce4c1cb5a16438786d70769154211f0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/4ce4c1cb5a16438786d70769154211f0 2023-02-13 12:25:50,017 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5d424be18d07d0a984d49d2450dc4b7e/family1 of 5d424be18d07d0a984d49d2450dc4b7e into 4ce4c1cb5a16438786d70769154211f0(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:50,017 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5d424be18d07d0a984d49d2450dc4b7e: 2023-02-13 12:25:50,017 INFO [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e., storeName=5d424be18d07d0a984d49d2450dc4b7e/family1, priority=12, startTime=1676291147579; duration=1sec 2023-02-13 12:25:50,017 DEBUG [RS:0;jenkins-hbase16:38471-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:50,022 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154] to archive 2023-02-13 12:25:50,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:50,025 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-8c9f1917816640868a9631ceb1e18a41 2023-02-13 12:25:50,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-05f2875b409d4605b82237bc61edc387 2023-02-13 12:25:50,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-f63b749fb7484be5b9fd853163640e64 2023-02-13 12:25:50,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/family1/testCompactionWithHFileLink=5d424be18d07d0a984d49d2450dc4b7e-152389f341ff4965a4002bff15921154 2023-02-13 12:25:50,146 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,146 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,146 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 4993132286c530710e08273860fde1c5, disabling compactions & flushes 2023-02-13 12:25:50,146 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,146 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. after waiting 0 ms 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,146 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing c410f61e2e0cf5f28a4b5c4a521e8eb0, disabling compactions & flushes 2023-02-13 12:25:50,147 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. after waiting 0 ms 2023-02-13 12:25:50,147 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing be0d091bd879acd2c49cb81bc7c86ecc, disabling compactions & flushes 2023-02-13 12:25:50,148 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 4993132286c530710e08273860fde1c5: Waiting for close lock at 1676291150146Disabling compacts and flushes for region at 1676291150146Disabling writes for close at 1676291150147 (+1 ms)Writing region close event to WAL at 1676291150147Closed at 1676291150147 2023-02-13 12:25:50,147 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,148 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,148 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,148 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. after waiting 0 ms 2023-02-13 12:25:50,148 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,148 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,149 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,149 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: Waiting for close lock at 1676291150146Disabling compacts and flushes for region at 1676291150146Disabling writes for close at 1676291150147 (+1 ms)Writing region close event to WAL at 1676291150148 (+1 ms)Closed at 1676291150148 2023-02-13 12:25:50,149 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for be0d091bd879acd2c49cb81bc7c86ecc: Waiting for close lock at 1676291150146Disabling compacts and flushes for region at 1676291150146Disabling writes for close at 1676291150148 (+2 ms)Writing region close event to WAL at 1676291150149 (+1 ms)Closed at 1676291150149 2023-02-13 12:25:50,154 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:25:50,156 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291150155"}]},"ts":"1676291150155"} 2023-02-13 12:25:50,156 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291150155"}]},"ts":"1676291150155"} 2023-02-13 12:25:50,156 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676291150155"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291150155"}]},"ts":"1676291150155"} 2023-02-13 12:25:50,159 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 12:25:50,161 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:25:50,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291150161"}]},"ts":"1676291150161"} 2023-02-13 12:25:50,163 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-13 12:25:50,206 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4993132286c530710e08273860fde1c5, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=be0d091bd879acd2c49cb81bc7c86ecc, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c410f61e2e0cf5f28a4b5c4a521e8eb0, ASSIGN}] 2023-02-13 12:25:50,208 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=be0d091bd879acd2c49cb81bc7c86ecc, ASSIGN 2023-02-13 12:25:50,208 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4993132286c530710e08273860fde1c5, ASSIGN 2023-02-13 12:25:50,209 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c410f61e2e0cf5f28a4b5c4a521e8eb0, ASSIGN 2023-02-13 12:25:50,209 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=be0d091bd879acd2c49cb81bc7c86ecc, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:50,209 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4993132286c530710e08273860fde1c5, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:50,210 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c410f61e2e0cf5f28a4b5c4a521e8eb0, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:25:50,364 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=c410f61e2e0cf5f28a4b5c4a521e8eb0, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,364 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=4993132286c530710e08273860fde1c5, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,364 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=be0d091bd879acd2c49cb81bc7c86ecc, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,365 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150364"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291150364"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291150364"}]},"ts":"1676291150364"} 2023-02-13 12:25:50,365 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676291150364"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291150364"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291150364"}]},"ts":"1676291150364"} 2023-02-13 12:25:50,364 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150364"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291150364"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291150364"}]},"ts":"1676291150364"} 2023-02-13 12:25:50,368 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure 4993132286c530710e08273860fde1c5, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:50,369 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure be0d091bd879acd2c49cb81bc7c86ecc, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:50,370 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure c410f61e2e0cf5f28a4b5c4a521e8eb0, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:25:50,427 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/.tmp/family2/0826bd0eb7fa458b91be659bf50ff6aa as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/0826bd0eb7fa458b91be659bf50ff6aa 2023-02-13 12:25:50,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:50,435 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 534c6c8744730b7c32dbd5f29d42ddff/family2 of 534c6c8744730b7c32dbd5f29d42ddff into 0826bd0eb7fa458b91be659bf50ff6aa(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:25:50,436 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 534c6c8744730b7c32dbd5f29d42ddff: 2023-02-13 12:25:50,436 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff., storeName=534c6c8744730b7c32dbd5f29d42ddff/family2, priority=12, startTime=1676291147526; duration=1sec 2023-02-13 12:25:50,436 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:25:50,520 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed] to archive 2023-02-13 12:25:50,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:25:50,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-ff51486288cf40ea9a1a39ac978b4141 2023-02-13 12:25:50,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4993132286c530710e08273860fde1c5, NAME => 'ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 12:25:50,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-4caed0ca74854af3b5159470b6045d40 2023-02-13 12:25:50,528 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-67a2ab442e3f4c0d838f9712651614c6 2023-02-13 12:25:50,530 DEBUG [StoreOpener-4993132286c530710e08273860fde1c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1 2023-02-13 12:25:50,530 DEBUG [StoreOpener-4993132286c530710e08273860fde1c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1 2023-02-13 12:25:50,531 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4993132286c530710e08273860fde1c5 columnFamilyName family1 2023-02-13 12:25:50,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/family2/testCompactionWithHFileLink=534c6c8744730b7c32dbd5f29d42ddff-948bf581acc3484aac27dd70302439ed 2023-02-13 12:25:50,532 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] regionserver.HStore(310): Store=4993132286c530710e08273860fde1c5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,533 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,534 DEBUG [StoreOpener-4993132286c530710e08273860fde1c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2 2023-02-13 12:25:50,534 DEBUG [StoreOpener-4993132286c530710e08273860fde1c5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2 2023-02-13 12:25:50,535 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4993132286c530710e08273860fde1c5 columnFamilyName family2 2023-02-13 12:25:50,536 INFO [StoreOpener-4993132286c530710e08273860fde1c5-1] regionserver.HStore(310): Store=4993132286c530710e08273860fde1c5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:50,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 4993132286c530710e08273860fde1c5 2023-02-13 12:25:50,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:50,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 4993132286c530710e08273860fde1c5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72553350, jitterRate=0.08112916350364685}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:50,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 4993132286c530710e08273860fde1c5: Running coprocessor pre-open hook at 1676291150526Writing region info on filesystem at 1676291150526Initializing all the Stores at 1676291150527 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150527Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150528 (+1 ms)Cleaning up temporary data from old regions at 1676291150539 (+11 ms)Cleaning up detritus from prior splits at 1676291150539Running coprocessor post-open hooks at 1676291150544 (+5 ms)Region opened successfully at 1676291150544 2023-02-13 12:25:50,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5., pid=109, masterSystemTime=1676291150520 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:50,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c410f61e2e0cf5f28a4b5c4a521e8eb0, NAME => 'ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 12:25:50,546 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=4993132286c530710e08273860fde1c5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,546 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150546"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291150546"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291150546"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291150546"}]},"ts":"1676291150546"} 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,548 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,549 DEBUG [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 2023-02-13 12:25:50,549 DEBUG [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 2023-02-13 12:25:50,549 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-02-13 12:25:50,549 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure 4993132286c530710e08273860fde1c5, server=jenkins-hbase16.apache.org,38471,1676291011293 in 179 msec 2023-02-13 12:25:50,549 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c410f61e2e0cf5f28a4b5c4a521e8eb0 columnFamilyName family1 2023-02-13 12:25:50,550 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=4993132286c530710e08273860fde1c5, ASSIGN in 343 msec 2023-02-13 12:25:50,550 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] regionserver.HStore(310): Store=c410f61e2e0cf5f28a4b5c4a521e8eb0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,551 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,552 DEBUG [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2 2023-02-13 12:25:50,552 DEBUG [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2 2023-02-13 12:25:50,552 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c410f61e2e0cf5f28a4b5c4a521e8eb0 columnFamilyName family2 2023-02-13 12:25:50,553 INFO [StoreOpener-c410f61e2e0cf5f28a4b5c4a521e8eb0-1] regionserver.HStore(310): Store=c410f61e2e0cf5f28a4b5c4a521e8eb0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:50,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:25:50,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:50,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened c410f61e2e0cf5f28a4b5c4a521e8eb0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59874973, jitterRate=-0.10779337584972382}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:50,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: Running coprocessor pre-open hook at 1676291150546Writing region info on filesystem at 1676291150546Initializing all the Stores at 1676291150547 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150547Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150547Cleaning up temporary data from old regions at 1676291150556 (+9 ms)Cleaning up detritus from prior splits at 1676291150556Running coprocessor post-open hooks at 1676291150562 (+6 ms)Region opened successfully at 1676291150562 2023-02-13 12:25:50,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0., pid=111, masterSystemTime=1676291150520 2023-02-13 12:25:50,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:50,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => be0d091bd879acd2c49cb81bc7c86ecc, NAME => 'ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 12:25:50,565 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=c410f61e2e0cf5f28a4b5c4a521e8eb0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,565 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676291150565"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291150565"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291150565"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291150565"}]},"ts":"1676291150565"} 2023-02-13 12:25:50,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:25:50,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,567 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,568 DEBUG [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1 2023-02-13 12:25:50,568 DEBUG [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1 2023-02-13 12:25:50,568 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-02-13 12:25:50,569 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure c410f61e2e0cf5f28a4b5c4a521e8eb0, server=jenkins-hbase16.apache.org,38471,1676291011293 in 197 msec 2023-02-13 12:25:50,569 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region be0d091bd879acd2c49cb81bc7c86ecc columnFamilyName family1 2023-02-13 12:25:50,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c410f61e2e0cf5f28a4b5c4a521e8eb0, ASSIGN in 363 msec 2023-02-13 12:25:50,570 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] regionserver.HStore(310): Store=be0d091bd879acd2c49cb81bc7c86ecc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,570 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,571 DEBUG [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2 2023-02-13 12:25:50,571 DEBUG [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2 2023-02-13 12:25:50,571 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region be0d091bd879acd2c49cb81bc7c86ecc columnFamilyName family2 2023-02-13 12:25:50,572 INFO [StoreOpener-be0d091bd879acd2c49cb81bc7c86ecc-1] regionserver.HStore(310): Store=be0d091bd879acd2c49cb81bc7c86ecc/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:25:50,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:25:50,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:25:50,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:25:50,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened be0d091bd879acd2c49cb81bc7c86ecc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72134806, jitterRate=0.07489237189292908}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:25:50,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for be0d091bd879acd2c49cb81bc7c86ecc: Running coprocessor pre-open hook at 1676291150566Writing region info on filesystem at 1676291150566Initializing all the Stores at 1676291150567 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150567Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291150567Cleaning up temporary data from old regions at 1676291150573 (+6 ms)Cleaning up detritus from prior splits at 1676291150573Running coprocessor post-open hooks at 1676291150576 (+3 ms)Region opened successfully at 1676291150576 2023-02-13 12:25:50,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc., pid=110, masterSystemTime=1676291150520 2023-02-13 12:25:50,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:50,578 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=be0d091bd879acd2c49cb81bc7c86ecc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:50,578 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676291150578"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291150578"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291150578"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291150578"}]},"ts":"1676291150578"} 2023-02-13 12:25:50,580 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-02-13 12:25:50,580 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure be0d091bd879acd2c49cb81bc7c86ecc, server=jenkins-hbase16.apache.org,38471,1676291011293 in 210 msec 2023-02-13 12:25:50,581 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-02-13 12:25:50,582 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=be0d091bd879acd2c49cb81bc7c86ecc, ASSIGN in 374 msec 2023-02-13 12:25:50,582 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:25:50,582 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291150582"}]},"ts":"1676291150582"} 2023-02-13 12:25:50,583 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-13 12:25:50,620 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:25:50,623 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.3000 sec 2023-02-13 12:25:51,374 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-13 12:25:51,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 12:25:51,435 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-13 12:25:51,447 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:51,448 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:51,449 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:51,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:51,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 12:25:51,454 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 12:25:51,454 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:51,454 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 12:25:51,454 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 12:25:51,454 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:51,455 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:51,496 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:51,496 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:51,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:51,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:51,497 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:51,497 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:51,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:51,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:51,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:51,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:51,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 12:25:51,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 12:25:51,499 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 12:25:51,500 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:51,500 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 12:25:51,500 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:51,500 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. started... 2023-02-13 12:25:51,500 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:51,501 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. started... 2023-02-13 12:25:51,501 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:51,504 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:51,505 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. started... 2023-02-13 12:25:51,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:51,505 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c410f61e2e0cf5f28a4b5c4a521e8eb0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:51,504 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing be0d091bd879acd2c49cb81bc7c86ecc 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:51,504 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4993132286c530710e08273860fde1c5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:25:51,525 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213d5a46c70f6024823bbdd34fa24574ade to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d5a46c70f6024823bbdd34fa24574ade 2023-02-13 12:25:51,525 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213acbf69a6e9d9489c9c25bf0fd542e14a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213acbf69a6e9d9489c9c25bf0fd542e14a 2023-02-13 12:25:51,933 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302135189f9559d60429fa7ec8c30da8ea65f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302135189f9559d60429fa7ec8c30da8ea65f 2023-02-13 12:25:51,934 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/ad59314c8b01416181ccad20098d8652 2023-02-13 12:25:51,934 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/0a14d4454bd74ad39ed53b84ddbce0ec 2023-02-13 12:25:52,341 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/a797cadff5a04e4b98fac4dfed7a62e9 2023-02-13 12:25:52,354 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302135663406320d14efabb83cfdbecba55ae to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302135663406320d14efabb83cfdbecba55ae 2023-02-13 12:25:52,354 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213ed8a8a146edb479ba3cc416a8daf3459 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213ed8a8a146edb479ba3cc416a8daf3459 2023-02-13 12:25:52,361 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/53e798cdc2dc435eaed5f8bd9c0ed882 2023-02-13 12:25:52,364 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/ad59314c8b01416181ccad20098d8652 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652 2023-02-13 12:25:52,367 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:52,367 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/53e798cdc2dc435eaed5f8bd9c0ed882 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882 2023-02-13 12:25:52,371 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:52,371 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4993132286c530710e08273860fde1c5 in 867ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:52,372 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:25:52,372 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:52,372 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:52,760 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/cc2c22317ab849469ff20bb97e151baa 2023-02-13 12:25:52,770 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213b4c8461f002b4a8cb0173003af8e6a07 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213b4c8461f002b4a8cb0173003af8e6a07 2023-02-13 12:25:52,772 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/0a14d4454bd74ad39ed53b84ddbce0ec as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec 2023-02-13 12:25:52,777 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c 2023-02-13 12:25:52,777 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:52,778 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/cc2c22317ab849469ff20bb97e151baa as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa 2023-02-13 12:25:52,782 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/a797cadff5a04e4b98fac4dfed7a62e9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9 2023-02-13 12:25:52,783 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:52,784 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c410f61e2e0cf5f28a4b5c4a521e8eb0 in 1279ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:52,784 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:25:52,784 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:52,784 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:52,785 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9, entries=4, sequenceid=5, filesize=5.4 K 2023-02-13 12:25:52,786 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c 2023-02-13 12:25:52,789 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:25:52,789 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for be0d091bd879acd2c49cb81bc7c86ecc in 1285ms, sequenceid=5, compaction requested=false 2023-02-13 12:25:52,790 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:25:52,790 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:52,790 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:52,790 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:52,790 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,790 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 12:25:52,790 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 12:25:52,823 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,823 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:52,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:52,823 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,823 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:52,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:52,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:52,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:52,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 12:25:52,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60ab682c[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:52,825 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 12:25:52,825 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,831 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 12:25:52,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 12:25:52,831 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:52,831 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,831 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:52,839 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 12:25:52,839 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,839 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:52,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,839 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 12:25:52,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:52,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:52,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:52,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:52,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:52,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:52,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 12:25:52,842 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 12:25:52,842 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:52,842 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:52,842 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 12:25:52,842 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:52,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:52,852 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:52,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:52,853 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,854 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:52,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 12:25:52,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:52,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:52,864 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:52,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 12:25:52,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:52,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:52,965 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:52,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:52,967 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:52,968 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:52,968 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:52,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:52,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:52,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 12:25:52,973 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 12:25:52,973 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:52,973 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 12:25:52,973 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 12:25:52,974 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,974 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:52,981 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:52,981 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:52,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:52,982 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:52,982 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:52,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:52,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:52,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:52,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:52,983 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 12:25:52,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 12:25:52,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 12:25:52,984 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 12:25:52,984 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:52,984 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 12:25:52,984 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:52,984 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. started... 2023-02-13 12:25:52,984 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:52,984 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:52,985 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4993132286c530710e08273860fde1c5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:52,984 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:52,985 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:52,985 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. started... 2023-02-13 12:25:52,985 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. started... 2023-02-13 12:25:52,985 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing be0d091bd879acd2c49cb81bc7c86ecc 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:52,985 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c410f61e2e0cf5f28a4b5c4a521e8eb0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:25:53,407 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021314977f0911f24bba90e6d7f5658d842c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021314977f0911f24bba90e6d7f5658d842c 2023-02-13 12:25:53,407 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021389a68f10c6464467a0a604064bd00b4d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021389a68f10c6464467a0a604064bd00b4d 2023-02-13 12:25:53,407 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213aae5d1cb1e0f45e0ad734bdfecd99bbd to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213aae5d1cb1e0f45e0ad734bdfecd99bbd 2023-02-13 12:25:53,412 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/e8517c7e1cb54158a2d3c43540c9cfea 2023-02-13 12:25:53,813 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/88b79afb405d4e549cfc8f165f9943a9 2023-02-13 12:25:53,818 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/acb1e7818da04ad99de3a2cada53647b 2023-02-13 12:25:53,825 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213992605bedacc4470abfe9f5abe79f002 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213992605bedacc4470abfe9f5abe79f002 2023-02-13 12:25:53,831 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/5a210c07a93c40a5b12f9ac8f1bfd71f 2023-02-13 12:25:53,834 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213337d006e3e9347f198edac80378f5a9d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213337d006e3e9347f198edac80378f5a9d 2023-02-13 12:25:53,835 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/e8517c7e1cb54158a2d3c43540c9cfea as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea 2023-02-13 12:25:53,838 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:53,839 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/5a210c07a93c40a5b12f9ac8f1bfd71f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f 2023-02-13 12:25:53,842 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:53,843 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c410f61e2e0cf5f28a4b5c4a521e8eb0 in 858ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:53,843 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:25:53,843 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:53,843 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:54,240 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/ab8605c81e5f45eda05439a4a53d448f 2023-02-13 12:25:54,244 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302139fa12d83bb9246338c74502c659b4bb8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302139fa12d83bb9246338c74502c659b4bb8 2023-02-13 12:25:54,248 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/88b79afb405d4e549cfc8f165f9943a9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9 2023-02-13 12:25:54,253 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:54,254 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/ab8605c81e5f45eda05439a4a53d448f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f 2023-02-13 12:25:54,258 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:54,258 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for be0d091bd879acd2c49cb81bc7c86ecc in 1273ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:54,259 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:25:54,259 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:54,259 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:54,656 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/172ba8ca81e2402784bcc426428eae93 2023-02-13 12:25:54,668 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/acb1e7818da04ad99de3a2cada53647b as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b 2023-02-13 12:25:54,673 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b, entries=4, sequenceid=9, filesize=5.4 K 2023-02-13 12:25:54,674 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/172ba8ca81e2402784bcc426428eae93 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93 2023-02-13 12:25:54,679 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:25:54,681 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4993132286c530710e08273860fde1c5 in 1697ms, sequenceid=9, compaction requested=false 2023-02-13 12:25:54,681 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:25:54,681 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:54,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:54,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:54,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 12:25:54,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 12:25:54,738 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,738 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:54,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:54,739 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,739 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:54,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:54,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:54,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:54,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 12:25:54,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@242633fb[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:54,740 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 12:25:54,740 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,752 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,753 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 12:25:54,753 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 12:25:54,753 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:54,753 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,753 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:54,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 12:25:54,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:54,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 12:25:54,761 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:54,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:54,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:54,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:54,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 12:25:54,765 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 12:25:54,765 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:54,765 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:54,765 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 12:25:54,765 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:54,773 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,773 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,773 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:54,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:54,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:54,773 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:54,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,775 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:54,786 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:54,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:54,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:54,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:54,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 12:25:54,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:54,786 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,787 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,787 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:54,787 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:54,787 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:54,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:54,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:54,787 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,888 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:54,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:54,892 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:54,893 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:54,893 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:54,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:54,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:54,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 12:25:54,902 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 12:25:54,902 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:54,902 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 12:25:54,902 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 12:25:54,903 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,903 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,911 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:54,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:54,911 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:54,912 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:54,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:54,912 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 12:25:54,912 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 12:25:54,912 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 12:25:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 12:25:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 12:25:54,912 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:54,912 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. started... 2023-02-13 12:25:54,913 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:54,913 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. started... 2023-02-13 12:25:54,912 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:54,913 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4993132286c530710e08273860fde1c5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:54,913 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c410f61e2e0cf5f28a4b5c4a521e8eb0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:54,912 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:54,913 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. started... 2023-02-13 12:25:54,913 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:54,913 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing be0d091bd879acd2c49cb81bc7c86ecc 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:25:54,923 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021375be2bea7c1d4da0ab8ae9ebead6feb6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021375be2bea7c1d4da0ab8ae9ebead6feb6 2023-02-13 12:25:55,328 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/61c96cf9bd7e4bc598675316bf0adcb3 2023-02-13 12:25:55,332 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021336cde0c8c5cb4b6fb6da08eaeec11e06 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021336cde0c8c5cb4b6fb6da08eaeec11e06 2023-02-13 12:25:55,332 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302139fa775b812bb4564ad0ec5ef9ec45640 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302139fa775b812bb4564ad0ec5ef9ec45640 2023-02-13 12:25:55,338 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/1532abe982ff4bb588adb9d07eb9e46a 2023-02-13 12:25:55,739 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/f448f555f2b74b71ba4c4242379dba07 2023-02-13 12:25:55,750 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e2b89e795d3041c086001bc8f022fd22 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230213e2b89e795d3041c086001bc8f022fd22 2023-02-13 12:25:55,752 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213d1cbd4435cfd41b29db92cc409522e3e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213d1cbd4435cfd41b29db92cc409522e3e 2023-02-13 12:25:55,756 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/087b71b383d04c57a6acf984ea3280f3 2023-02-13 12:25:55,759 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/1532abe982ff4bb588adb9d07eb9e46a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a 2023-02-13 12:25:55,763 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:55,764 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/087b71b383d04c57a6acf984ea3280f3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3 2023-02-13 12:25:55,768 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:55,769 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for be0d091bd879acd2c49cb81bc7c86ecc in 856ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:55,769 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:25:55,769 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:55,769 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:56,030 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:25:56,156 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/3b51089638664ffba6da26b21e19a86c 2023-02-13 12:25:56,163 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213755ff9a76f304c9b831b6b5b97628d62 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213755ff9a76f304c9b831b6b5b97628d62 2023-02-13 12:25:56,164 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/61c96cf9bd7e4bc598675316bf0adcb3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3 2023-02-13 12:25:56,169 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:56,170 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/3b51089638664ffba6da26b21e19a86c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c 2023-02-13 12:25:56,174 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:56,175 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4993132286c530710e08273860fde1c5 in 1262ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:56,175 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:25:56,175 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:56,175 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:56,571 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/09d6301447b04538a16ca548efdb77e6 2023-02-13 12:25:56,585 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/f448f555f2b74b71ba4c4242379dba07 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07 2023-02-13 12:25:56,591 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07, entries=4, sequenceid=13, filesize=5.4 K 2023-02-13 12:25:56,592 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/09d6301447b04538a16ca548efdb77e6 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6 2023-02-13 12:25:56,598 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:25:56,599 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c410f61e2e0cf5f28a4b5c4a521e8eb0 in 1686ms, sequenceid=13, compaction requested=true 2023-02-13 12:25:56,599 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:25:56,599 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:56,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:56,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:56,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 12:25:56,599 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 12:25:56,644 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,644 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:56,645 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,646 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:56,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:56,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:56,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:56,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 12:25:56,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63381ebf[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:56,651 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 12:25:56,652 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,661 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 12:25:56,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 12:25:56,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:56,662 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,662 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:56,673 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 12:25:56,673 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:56,673 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,673 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 12:25:56,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:56,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:56,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:56,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:56,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:56,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:56,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 12:25:56,683 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 12:25:56,683 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:56,683 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:56,683 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 12:25:56,683 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:56,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:56,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:56,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:56,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:56,696 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:56,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,700 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:56,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:56,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:56,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:56,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:56,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:56,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 12:25:56,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:56,714 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:56,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:56,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 12:25:56,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:56,714 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:56,715 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,815 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:56,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:56,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:56,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:56,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:25:56,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:56,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:56,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 12:25:56,826 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 12:25:56,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:56,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 12:25:56,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 12:25:56,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,827 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:56,835 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,835 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:56,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:56,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:56,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:56,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:56,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:56,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:56,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:56,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:56,838 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 12:25:56,838 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 12:25:56,838 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 12:25:56,838 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 12:25:56,838 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:56,838 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 12:25:56,838 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:56,838 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:56,839 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:56,839 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:56,839 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. started... 2023-02-13 12:25:56,838 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. started... 2023-02-13 12:25:56,839 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c410f61e2e0cf5f28a4b5c4a521e8eb0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:56,839 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:56,839 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. started... 2023-02-13 12:25:56,839 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4993132286c530710e08273860fde1c5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:56,839 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing be0d091bd879acd2c49cb81bc7c86ecc 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:25:56,856 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213294ecdbf431f4681b6c8d4ebd8fb112e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213294ecdbf431f4681b6c8d4ebd8fb112e 2023-02-13 12:25:57,262 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/b55e2d68341a48d08cb3db1e15d6d6f8 2023-02-13 12:25:57,263 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302134b19f91e4a064671b804678ead5a8c03 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302134b19f91e4a064671b804678ead5a8c03 2023-02-13 12:25:57,263 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213f0fa10f6309e4f14810bf897a802a545 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213f0fa10f6309e4f14810bf897a802a545 2023-02-13 12:25:57,670 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/43670c5ebc364f06a1fa5aa9ae902768 2023-02-13 12:25:57,670 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/cd06f6ee9b6c40cd909d87001a3f1f8e 2023-02-13 12:25:57,681 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021357f7f3698afe4910b97bd6e4021ace8a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023021357f7f3698afe4910b97bd6e4021ace8a 2023-02-13 12:25:58,090 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/82452d09fd014c479802d95e322e4589 2023-02-13 12:25:58,100 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213202bbe8a246e441996ae6db5873b1072 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230213202bbe8a246e441996ae6db5873b1072 2023-02-13 12:25:58,100 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021373a621dbed244729a7b4b65bf1a20cd6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023021373a621dbed244729a7b4b65bf1a20cd6 2023-02-13 12:25:58,100 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/b55e2d68341a48d08cb3db1e15d6d6f8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8 2023-02-13 12:25:58,105 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:58,105 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/82452d09fd014c479802d95e322e4589 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589 2023-02-13 12:25:58,109 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:58,110 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c410f61e2e0cf5f28a4b5c4a521e8eb0 in 1271ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:58,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:25:58,110 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:58,110 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:25:58,506 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/48319bf55c7b4022b4a5bdbb9e1b078c 2023-02-13 12:25:58,506 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/a6da5cf74c814b8abebf262b92fe2044 2023-02-13 12:25:58,517 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/cd06f6ee9b6c40cd909d87001a3f1f8e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e 2023-02-13 12:25:58,518 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/43670c5ebc364f06a1fa5aa9ae902768 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768 2023-02-13 12:25:58,522 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:58,523 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768, entries=4, sequenceid=17, filesize=5.4 K 2023-02-13 12:25:58,523 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/48319bf55c7b4022b4a5bdbb9e1b078c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c 2023-02-13 12:25:58,523 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/a6da5cf74c814b8abebf262b92fe2044 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044 2023-02-13 12:25:58,528 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:58,528 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:25:58,528 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4993132286c530710e08273860fde1c5 in 1689ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:58,528 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for be0d091bd879acd2c49cb81bc7c86ecc in 1689ms, sequenceid=17, compaction requested=true 2023-02-13 12:25:58,528 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:25:58,529 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:25:58,529 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:58,529 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 12:25:58,529 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 12:25:58,560 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,560 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:58,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:58,561 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,561 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:25:58,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:58,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:58,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:58,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 12:25:58,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e8e1b23[Count = 0] remaining members to acquire global barrier 2023-02-13 12:25:58,563 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 12:25:58,563 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,611 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 12:25:58,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 12:25:58,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:25:58,612 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,612 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:25:58,664 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 12:25:58,665 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:58,665 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 12:25:58,665 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:58,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:58,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:58,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:58,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:58,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:25:58,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 12:25:58,676 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 12:25:58,676 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:25:58,676 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:25:58,676 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 12:25:58,676 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:25:58,685 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,685 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,686 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:58,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:25:58,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:25:58,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:25:58,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:58,687 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:58,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:25:58,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,690 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:25:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:25:58,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,702 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:58,702 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:58,702 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:25:58,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 12:25:58,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:25:58,702 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:25:58,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:58,702 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:25:58,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:25:58,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:25:58,703 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,803 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:25:58,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:25:58,822 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:58,833 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:58,842 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:25:58,842 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:25:58,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:25:58,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-13 12:25:58,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-13 12:25:58,874 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-13 12:25:58,874 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:58,874 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-13 12:25:58,874 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-13 12:25:58,874 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,874 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:58,881 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,881 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:25:58,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:25:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:25:58,881 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:25:58,881 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:25:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:58,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:25:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:25:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:25:58,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-13 12:25:58,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-13 12:25:58,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-13 12:25:58,882 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-13 12:25:58,882 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:25:58,882 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-13 12:25:58,882 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:25:58,882 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. started... 2023-02-13 12:25:58,882 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:25:58,883 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:25:58,883 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4993132286c530710e08273860fde1c5 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:58,883 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:25:58,883 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. started... 2023-02-13 12:25:58,883 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:25:58,883 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. started... 2023-02-13 12:25:58,883 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing be0d091bd879acd2c49cb81bc7c86ecc 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:58,883 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c410f61e2e0cf5f28a4b5c4a521e8eb0 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:25:59,317 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/897bba0e3e0648feb10d266e6da825df 2023-02-13 12:25:59,317 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/749d8f517c224465a4f7ae379014b3eb 2023-02-13 12:25:59,322 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 897bba0e3e0648feb10d266e6da825df 2023-02-13 12:25:59,323 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 749d8f517c224465a4f7ae379014b3eb 2023-02-13 12:25:59,722 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1 2023-02-13 12:25:59,732 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54ffb5a7a78746ae8c4a0bc8261bcbd1 2023-02-13 12:26:00,138 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/72ae1174f4464e5c815035dcbec317f6 2023-02-13 12:26:00,144 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/52d129a29f8e4fc0af899b16441708f3 2023-02-13 12:26:00,147 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ae1174f4464e5c815035dcbec317f6 2023-02-13 12:26:00,148 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/897bba0e3e0648feb10d266e6da825df as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df 2023-02-13 12:26:00,153 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 52d129a29f8e4fc0af899b16441708f3 2023-02-13 12:26:00,153 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/749d8f517c224465a4f7ae379014b3eb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb 2023-02-13 12:26:00,155 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 897bba0e3e0648feb10d266e6da825df 2023-02-13 12:26:00,155 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,156 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/72ae1174f4464e5c815035dcbec317f6 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6 2023-02-13 12:26:00,157 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 749d8f517c224465a4f7ae379014b3eb 2023-02-13 12:26:00,157 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,158 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/52d129a29f8e4fc0af899b16441708f3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3 2023-02-13 12:26:00,160 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72ae1174f4464e5c815035dcbec317f6 2023-02-13 12:26:00,160 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,161 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c410f61e2e0cf5f28a4b5c4a521e8eb0 in 1278ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:00,161 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:26:00,161 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:00,161 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:00,161 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 52d129a29f8e4fc0af899b16441708f3 2023-02-13 12:26:00,161 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,161 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4993132286c530710e08273860fde1c5 in 1278ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:00,162 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:26:00,162 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:00,162 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:00,554 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/4ec04d678d1a4a4f8eff17e02f19a234 2023-02-13 12:26:00,565 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4ec04d678d1a4a4f8eff17e02f19a234 2023-02-13 12:26:00,566 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1 2023-02-13 12:26:00,570 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54ffb5a7a78746ae8c4a0bc8261bcbd1 2023-02-13 12:26:00,570 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1, entries=3, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,570 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/4ec04d678d1a4a4f8eff17e02f19a234 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234 2023-02-13 12:26:00,573 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4ec04d678d1a4a4f8eff17e02f19a234 2023-02-13 12:26:00,574 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:00,574 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for be0d091bd879acd2c49cb81bc7c86ecc in 1691ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:00,574 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:26:00,574 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:00,574 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:00,574 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:00,575 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,575 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-13 12:26:00,575 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-13 12:26:00,611 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:00,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:00,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,611 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:00,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:00,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:00,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:00,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-13 12:26:00,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2de031ab[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:00,614 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-13 12:26:00,614 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,645 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,645 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-13 12:26:00,645 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-13 12:26:00,645 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:00,646 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,646 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:00,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-13 12:26:00,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:00,661 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-13 12:26:00,661 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:00,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:00,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:00,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:00,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:00,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:00,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-13 12:26:00,671 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-13 12:26:00,671 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:00,671 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:00,671 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-13 12:26:00,671 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:00,681 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,681 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,681 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:00,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:00,682 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:00,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,684 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:00,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-13 12:26:00,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-13 12:26:00,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:00,694 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-13 12:26:00,694 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:00,695 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-13 12:26:00,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:00,695 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:00,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:00,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-13 12:26:00,695 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:00,696 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:00,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:00,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:00,796 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:00,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:00,798 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:00,798 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 4993132286c530710e08273860fde1c5/family1 is initiating major compaction (all files) 2023-02-13 12:26:00,798 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:00,799 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:00,799 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 4993132286c530710e08273860fde1c5/family1 in ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:00,799 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp, totalSize=26.6 K 2023-02-13 12:26:00,800 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting ad59314c8b01416181ccad20098d8652, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151447 2023-02-13 12:26:00,801 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting acb1e7818da04ad99de3a2cada53647b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152967 2023-02-13 12:26:00,802 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 61c96cf9bd7e4bc598675316bf0adcb3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154892 2023-02-13 12:26:00,803 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting cd06f6ee9b6c40cd909d87001a3f1f8e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:00,804 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 749d8f517c224465a4f7ae379014b3eb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:00,817 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 4993132286c530710e08273860fde1c5#family1#compaction#150 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:01,233 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021352163caac42742f7a90342e2b2d685a6_del 2023-02-13 12:26:01,233 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021352163caac42742f7a90342e2b2d685a6_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021352163caac42742f7a90342e2b2d685a6_del 2023-02-13 12:26:02,068 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ccbaf6ac2bb40d1ba1b899a82395c37 2023-02-13 12:26:02,070 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family1/7ccbaf6ac2bb40d1ba1b899a82395c37 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/7ccbaf6ac2bb40d1ba1b899a82395c37 2023-02-13 12:26:02,078 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ccbaf6ac2bb40d1ba1b899a82395c37 2023-02-13 12:26:02,079 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4993132286c530710e08273860fde1c5/family1 of 4993132286c530710e08273860fde1c5 into 7ccbaf6ac2bb40d1ba1b899a82395c37(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:02,079 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:26:02,079 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:02,079 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 4993132286c530710e08273860fde1c5/family2 is initiating major compaction (all files) 2023-02-13 12:26:02,080 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:02,080 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:02,080 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 4993132286c530710e08273860fde1c5/family2 in ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:02,080 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp, totalSize=25.3 K 2023-02-13 12:26:02,081 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 53e798cdc2dc435eaed5f8bd9c0ed882, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151447 2023-02-13 12:26:02,081 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 172ba8ca81e2402784bcc426428eae93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152967 2023-02-13 12:26:02,082 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 3b51089638664ffba6da26b21e19a86c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154892 2023-02-13 12:26:02,083 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 48319bf55c7b4022b4a5bdbb9e1b078c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:02,083 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 52d129a29f8e4fc0af899b16441708f3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:02,096 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 4993132286c530710e08273860fde1c5#family2#compaction#151 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:02,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb] to archive 2023-02-13 12:26:02,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:02,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ad59314c8b01416181ccad20098d8652 2023-02-13 12:26:02,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/acb1e7818da04ad99de3a2cada53647b 2023-02-13 12:26:02,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/61c96cf9bd7e4bc598675316bf0adcb3 2023-02-13 12:26:02,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/cd06f6ee9b6c40cd909d87001a3f1f8e 2023-02-13 12:26:02,133 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/749d8f517c224465a4f7ae379014b3eb 2023-02-13 12:26:02,514 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302138f45f28e4043427e909a35eae369bd66_del 2023-02-13 12:26:02,514 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302138f45f28e4043427e909a35eae369bd66_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302138f45f28e4043427e909a35eae369bd66_del 2023-02-13 12:26:03,340 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1422c9c07d584cd5bbd4ddd9d7fb0916 2023-02-13 12:26:03,342 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/.tmp/family2/1422c9c07d584cd5bbd4ddd9d7fb0916 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/1422c9c07d584cd5bbd4ddd9d7fb0916 2023-02-13 12:26:03,349 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1422c9c07d584cd5bbd4ddd9d7fb0916 2023-02-13 12:26:03,350 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4993132286c530710e08273860fde1c5/family2 of 4993132286c530710e08273860fde1c5 into 1422c9c07d584cd5bbd4ddd9d7fb0916(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:03,350 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:26:03,350 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:03,351 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): be0d091bd879acd2c49cb81bc7c86ecc/family1 is initiating major compaction (all files) 2023-02-13 12:26:03,351 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:03,351 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:03,351 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of be0d091bd879acd2c49cb81bc7c86ecc/family1 in ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:03,351 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp, totalSize=26.6 K 2023-02-13 12:26:03,352 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting a797cadff5a04e4b98fac4dfed7a62e9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151448 2023-02-13 12:26:03,352 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 88b79afb405d4e549cfc8f165f9943a9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152968 2023-02-13 12:26:03,353 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 1532abe982ff4bb588adb9d07eb9e46a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154893 2023-02-13 12:26:03,354 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 43670c5ebc364f06a1fa5aa9ae902768, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:03,354 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 54ffb5a7a78746ae8c4a0bc8261bcbd1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:03,366 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): be0d091bd879acd2c49cb81bc7c86ecc#family1#compaction#152 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:03,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3] to archive 2023-02-13 12:26:03,425 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:03,428 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/53e798cdc2dc435eaed5f8bd9c0ed882 2023-02-13 12:26:03,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/172ba8ca81e2402784bcc426428eae93 2023-02-13 12:26:03,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/3b51089638664ffba6da26b21e19a86c 2023-02-13 12:26:03,433 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/48319bf55c7b4022b4a5bdbb9e1b078c 2023-02-13 12:26:03,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family2/52d129a29f8e4fc0af899b16441708f3 2023-02-13 12:26:03,781 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021346748a90d4214255a69036b878bd3600_del 2023-02-13 12:26:03,781 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021346748a90d4214255a69036b878bd3600_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021346748a90d4214255a69036b878bd3600_del 2023-02-13 12:26:04,606 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb7f0f19b66b48e1ac7d3e1252a56c74 2023-02-13 12:26:04,608 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family1/cb7f0f19b66b48e1ac7d3e1252a56c74 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/cb7f0f19b66b48e1ac7d3e1252a56c74 2023-02-13 12:26:04,611 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb7f0f19b66b48e1ac7d3e1252a56c74 2023-02-13 12:26:04,612 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in be0d091bd879acd2c49cb81bc7c86ecc/family1 of be0d091bd879acd2c49cb81bc7c86ecc into cb7f0f19b66b48e1ac7d3e1252a56c74(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:04,612 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:26:04,612 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:04,612 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): be0d091bd879acd2c49cb81bc7c86ecc/family2 is initiating major compaction (all files) 2023-02-13 12:26:04,612 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:04,613 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:04,613 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of be0d091bd879acd2c49cb81bc7c86ecc/family2 in ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:04,613 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp, totalSize=25.3 K 2023-02-13 12:26:04,613 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f0aa6cfa5ed242b6a2fa600be67c9f5c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151448 2023-02-13 12:26:04,613 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting ab8605c81e5f45eda05439a4a53d448f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152968 2023-02-13 12:26:04,614 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 087b71b383d04c57a6acf984ea3280f3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154893 2023-02-13 12:26:04,614 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting a6da5cf74c814b8abebf262b92fe2044, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:04,614 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 4ec04d678d1a4a4f8eff17e02f19a234, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:04,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1] to archive 2023-02-13 12:26:04,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:04,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/a797cadff5a04e4b98fac4dfed7a62e9 2023-02-13 12:26:04,623 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): be0d091bd879acd2c49cb81bc7c86ecc#family2#compaction#153 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:04,623 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/88b79afb405d4e549cfc8f165f9943a9 2023-02-13 12:26:04,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/1532abe982ff4bb588adb9d07eb9e46a 2023-02-13 12:26:04,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/43670c5ebc364f06a1fa5aa9ae902768 2023-02-13 12:26:04,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/54ffb5a7a78746ae8c4a0bc8261bcbd1 2023-02-13 12:26:04,638 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:26:04,638 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dbcc7d9 to 127.0.0.1:64508 2023-02-13 12:26:04,638 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:04,639 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:26:04,639 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32e26b9f to 127.0.0.1:64508 2023-02-13 12:26:04,639 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:04,639 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021392ff670792994dfb8fb24ae9139c643a_del 2023-02-13 12:26:04,640 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021392ff670792994dfb8fb24ae9139c643a_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023021392ff670792994dfb8fb24ae9139c643a_del 2023-02-13 12:26:05,467 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c5b7f94d873435d82729e3c0902e6b2 2023-02-13 12:26:05,468 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/.tmp/family2/6c5b7f94d873435d82729e3c0902e6b2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/6c5b7f94d873435d82729e3c0902e6b2 2023-02-13 12:26:05,476 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c5b7f94d873435d82729e3c0902e6b2 2023-02-13 12:26:05,478 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in be0d091bd879acd2c49cb81bc7c86ecc/family2 of be0d091bd879acd2c49cb81bc7c86ecc into 6c5b7f94d873435d82729e3c0902e6b2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:26:05,478 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:26:05,478 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:05,479 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 is initiating major compaction (all files) 2023-02-13 12:26:05,479 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:05,479 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:05,480 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 in ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:05,480 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp, totalSize=26.6 K 2023-02-13 12:26:05,481 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 0a14d4454bd74ad39ed53b84ddbce0ec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151449 2023-02-13 12:26:05,481 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting e8517c7e1cb54158a2d3c43540c9cfea, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152968 2023-02-13 12:26:05,482 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f448f555f2b74b71ba4c4242379dba07, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154893 2023-02-13 12:26:05,482 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting b55e2d68341a48d08cb3db1e15d6d6f8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:05,483 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 897bba0e3e0648feb10d266e6da825df, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:05,493 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): c410f61e2e0cf5f28a4b5c4a521e8eb0#family1#compaction#154 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:05,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234] to archive 2023-02-13 12:26:05,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:05,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/f0aa6cfa5ed242b6a2fa600be67c9f5c 2023-02-13 12:26:05,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/ab8605c81e5f45eda05439a4a53d448f 2023-02-13 12:26:05,529 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/087b71b383d04c57a6acf984ea3280f3 2023-02-13 12:26:05,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/a6da5cf74c814b8abebf262b92fe2044 2023-02-13 12:26:05,532 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family2/4ec04d678d1a4a4f8eff17e02f19a234 2023-02-13 12:26:05,920 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213397f0fcbb0704e67864a4a1eab1647bc_del 2023-02-13 12:26:05,920 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213397f0fcbb0704e67864a4a1eab1647bc_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213397f0fcbb0704e67864a4a1eab1647bc_del 2023-02-13 12:26:06,755 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 650b767d230944c095022fd2806f6944 2023-02-13 12:26:06,757 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family1/650b767d230944c095022fd2806f6944 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/650b767d230944c095022fd2806f6944 2023-02-13 12:26:06,761 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 650b767d230944c095022fd2806f6944 2023-02-13 12:26:06,762 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 of c410f61e2e0cf5f28a4b5c4a521e8eb0 into 650b767d230944c095022fd2806f6944(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:06,762 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:26:06,762 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:06,762 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): c410f61e2e0cf5f28a4b5c4a521e8eb0/family2 is initiating major compaction (all files) 2023-02-13 12:26:06,762 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:06,762 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:06,762 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of c410f61e2e0cf5f28a4b5c4a521e8eb0/family2 in ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:06,763 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp, totalSize=25.3 K 2023-02-13 12:26:06,763 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting cc2c22317ab849469ff20bb97e151baa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291151449 2023-02-13 12:26:06,763 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 5a210c07a93c40a5b12f9ac8f1bfd71f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291152968 2023-02-13 12:26:06,764 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 09d6301447b04538a16ca548efdb77e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291154893 2023-02-13 12:26:06,764 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 82452d09fd014c479802d95e322e4589, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291156818 2023-02-13 12:26:06,765 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 72ae1174f4464e5c815035dcbec317f6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:06,772 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): c410f61e2e0cf5f28a4b5c4a521e8eb0#family2#compaction#155 average throughput is 1.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:06,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df] to archive 2023-02-13 12:26:06,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:06,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/0a14d4454bd74ad39ed53b84ddbce0ec 2023-02-13 12:26:06,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/e8517c7e1cb54158a2d3c43540c9cfea 2023-02-13 12:26:06,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/f448f555f2b74b71ba4c4242379dba07 2023-02-13 12:26:06,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/b55e2d68341a48d08cb3db1e15d6d6f8 2023-02-13 12:26:06,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/897bba0e3e0648feb10d266e6da825df 2023-02-13 12:26:07,187 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213678c52549db8492691b15f8dab4bb52d_del 2023-02-13 12:26:07,187 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213678c52549db8492691b15f8dab4bb52d_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230213678c52549db8492691b15f8dab4bb52d_del 2023-02-13 12:26:08,017 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8235e2560884f3fa3b2acb5904e972e 2023-02-13 12:26:08,018 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/.tmp/family2/e8235e2560884f3fa3b2acb5904e972e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/e8235e2560884f3fa3b2acb5904e972e 2023-02-13 12:26:08,024 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e8235e2560884f3fa3b2acb5904e972e 2023-02-13 12:26:08,024 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c410f61e2e0cf5f28a4b5c4a521e8eb0/family2 of c410f61e2e0cf5f28a4b5c4a521e8eb0 into e8235e2560884f3fa3b2acb5904e972e(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:08,025 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:26:08,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6] to archive 2023-02-13 12:26:08,029 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:08,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:08,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/cc2c22317ab849469ff20bb97e151baa 2023-02-13 12:26:08,032 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/5a210c07a93c40a5b12f9ac8f1bfd71f 2023-02-13 12:26:08,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/09d6301447b04538a16ca548efdb77e6 2023-02-13 12:26:08,034 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:08,035 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/82452d09fd014c479802d95e322e4589 2023-02-13 12:26:08,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family2/72ae1174f4464e5c815035dcbec317f6 2023-02-13 12:26:08,038 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:08,038 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:08,043 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:26:08,055 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-13 12:26:08,057 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021352163caac42742f7a90342e2b2d685a6_del 2023-02-13 12:26:08,059 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021346748a90d4214255a69036b878bd3600_del 2023-02-13 12:26:08,062 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213397f0fcbb0704e67864a4a1eab1647bc_del 2023-02-13 12:26:08,062 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,063 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(139): Connect 0x29ece623 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:26:08,111 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@238115f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:26:08,113 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,113 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,113 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,546 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213df45af6658774e329d86d39038441718 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213df45af6658774e329d86d39038441718 2023-02-13 12:26:08,547 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,560 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:26:08,563 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:52708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:26:08,569 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:26:08,572 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:41722, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:26:08,584 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65999dea2feb47efb34d66f030da6c68 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-13 12:26:08,585 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65999dea2feb47efb34d66f030da6c68}] 2023-02-13 12:26:08,591 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65999dea2feb47efb34d66f030da6c68 for inclusion in 4993132286c530710e08273860fde1c5/family1 2023-02-13 12:26:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-13 12:26:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 12:26:08,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 4993132286c530710e08273860fde1c5: 2023-02-13 12:26:08,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65999dea2feb47efb34d66f030da6c68 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__kobn49rbbiidvhc9lfu8csg224b4t30g7nj1ildn45mo2h4b0rli4ucorp4441ma/family1/65999dea2feb47efb34d66f030da6c68 2023-02-13 12:26:08,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__kobn49rbbiidvhc9lfu8csg224b4t30g7nj1ildn45mo2h4b0rli4ucorp4441ma/family1/65999dea2feb47efb34d66f030da6c68 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ea7497a842cb442da7e9f9fee769bc84_SeqId_28_ 2023-02-13 12:26:08,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__kobn49rbbiidvhc9lfu8csg224b4t30g7nj1ildn45mo2h4b0rli4ucorp4441ma/family1/65999dea2feb47efb34d66f030da6c68 into 4993132286c530710e08273860fde1c5/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ea7497a842cb442da7e9f9fee769bc84_SeqId_28_ - updating store file list. 2023-02-13 12:26:08,598 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ea7497a842cb442da7e9f9fee769bc84_SeqId_28_ into 4993132286c530710e08273860fde1c5/family1 2023-02-13 12:26:08,598 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__kobn49rbbiidvhc9lfu8csg224b4t30g7nj1ildn45mo2h4b0rli4ucorp4441ma/family1/65999dea2feb47efb34d66f030da6c68 into 4993132286c530710e08273860fde1c5/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/family1/ea7497a842cb442da7e9f9fee769bc84_SeqId_28_) 2023-02-13 12:26:08,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__kobn49rbbiidvhc9lfu8csg224b4t30g7nj1ildn45mo2h4b0rli4ucorp4441ma/family1/65999dea2feb47efb34d66f030da6c68 2023-02-13 12:26:08,600 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,601 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:08,602 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213acbf69a6e9d9489c9c25bf0fd542e14a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230213acbf69a6e9d9489c9c25bf0fd542e14a 2023-02-13 12:26:08,602 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 12:26:08,602 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 12:26:08,602 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,946 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302131e162323766a428cbfa5483d92de50fa to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302131e162323766a428cbfa5483d92de50fa 2023-02-13 12:26:08,946 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213cf4b49883fac4a0abe1fd40e2b302131 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230213cf4b49883fac4a0abe1fd40e2b302131 2023-02-13 12:26:08,947 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,947 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,975 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/cced4e574e1c4ffa8fc7ec440e1a7542 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-13 12:26:08,975 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/ca2b43c5db234a1f89d036c4acb2808a first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-13 12:26:08,976 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/ca2b43c5db234a1f89d036c4acb2808a}] 2023-02-13 12:26:08,976 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/cced4e574e1c4ffa8fc7ec440e1a7542}] 2023-02-13 12:26:08,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/ca2b43c5db234a1f89d036c4acb2808a for inclusion in be0d091bd879acd2c49cb81bc7c86ecc/family1 2023-02-13 12:26:08,978 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/cced4e574e1c4ffa8fc7ec440e1a7542 for inclusion in c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: 2023-02-13 12:26:08,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for be0d091bd879acd2c49cb81bc7c86ecc: 2023-02-13 12:26:08,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213/family1/ca2b43c5db234a1f89d036c4acb2808a to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__lui53eag3cuc16oat0a5m8j56j697s0el41qjpck5fo2mo25f4f659dhfg6pppls/family1/ca2b43c5db234a1f89d036c4acb2808a 2023-02-13 12:26:08,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213/family1/cced4e574e1c4ffa8fc7ec440e1a7542 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__613q5qaquqp7q1fe8nbfi7cnj1ilb1dt3ltjdg4tfafne0i53rts67favc0g9jt9/family1/cced4e574e1c4ffa8fc7ec440e1a7542 2023-02-13 12:26:08,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__lui53eag3cuc16oat0a5m8j56j697s0el41qjpck5fo2mo25f4f659dhfg6pppls/family1/ca2b43c5db234a1f89d036c4acb2808a as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/e4299f6c280f46c79a920a70fe4e27f3_SeqId_28_ 2023-02-13 12:26:08,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__613q5qaquqp7q1fe8nbfi7cnj1ilb1dt3ltjdg4tfafne0i53rts67favc0g9jt9/family1/cced4e574e1c4ffa8fc7ec440e1a7542 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/011c9554a8b84b3dbd34061c9e4ff530_SeqId_28_ 2023-02-13 12:26:08,982 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__lui53eag3cuc16oat0a5m8j56j697s0el41qjpck5fo2mo25f4f659dhfg6pppls/family1/ca2b43c5db234a1f89d036c4acb2808a into be0d091bd879acd2c49cb81bc7c86ecc/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/e4299f6c280f46c79a920a70fe4e27f3_SeqId_28_ - updating store file list. 2023-02-13 12:26:08,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__613q5qaquqp7q1fe8nbfi7cnj1ilb1dt3ltjdg4tfafne0i53rts67favc0g9jt9/family1/cced4e574e1c4ffa8fc7ec440e1a7542 into c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/011c9554a8b84b3dbd34061c9e4ff530_SeqId_28_ - updating store file list. 2023-02-13 12:26:08,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/011c9554a8b84b3dbd34061c9e4ff530_SeqId_28_ into c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 2023-02-13 12:26:08,985 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/e4299f6c280f46c79a920a70fe4e27f3_SeqId_28_ into be0d091bd879acd2c49cb81bc7c86ecc/family1 2023-02-13 12:26:08,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__613q5qaquqp7q1fe8nbfi7cnj1ilb1dt3ltjdg4tfafne0i53rts67favc0g9jt9/family1/cced4e574e1c4ffa8fc7ec440e1a7542 into c410f61e2e0cf5f28a4b5c4a521e8eb0/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/family1/011c9554a8b84b3dbd34061c9e4ff530_SeqId_28_) 2023-02-13 12:26:08,985 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__lui53eag3cuc16oat0a5m8j56j697s0el41qjpck5fo2mo25f4f659dhfg6pppls/family1/ca2b43c5db234a1f89d036c4acb2808a into be0d091bd879acd2c49cb81bc7c86ecc/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/family1/e4299f6c280f46c79a920a70fe4e27f3_SeqId_28_) 2023-02-13 12:26:08,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__613q5qaquqp7q1fe8nbfi7cnj1ilb1dt3ltjdg4tfafne0i53rts67favc0g9jt9/family1/cced4e574e1c4ffa8fc7ec440e1a7542 2023-02-13 12:26:08,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__ns_testMinorCompaction__lui53eag3cuc16oat0a5m8j56j697s0el41qjpck5fo2mo25f4f659dhfg6pppls/family1/ca2b43c5db234a1f89d036c4acb2808a 2023-02-13 12:26:08,986 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,987 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,987 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:08,987 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:08,988 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302135189f9559d60429fa7ec8c30da8ea65f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302135189f9559d60429fa7ec8c30da8ea65f 2023-02-13 12:26:08,988 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d5a46c70f6024823bbdd34fa24574ade to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230213d5a46c70f6024823bbdd34fa24574ade 2023-02-13 12:26:08,988 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 12:26:08,988 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 12:26:08,988 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-13 12:26:08,988 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 12:26:08,989 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,989 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='ns:testMinorCompaction' and column='family1' 2023-02-13 12:26:08,989 INFO [Listener at localhost.localdomain/33629] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-13 12:26:08,992 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:08,994 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:08,997 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:08,998 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:09,016 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=246 (was 275), OpenFileDescriptor=613 (was 745), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=47 (was 48), ProcessCount=167 (was 167), AvailableMemoryMB=5872 (was 5881) 2023-02-13 12:26:09,025 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=246, OpenFileDescriptor=613, MaxFileDescriptor=60000, SystemLoadAverage=47, ProcessCount=167, AvailableMemoryMB=5872 2023-02-13 12:26:09,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-13 12:26:09,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:09,028 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:26:09,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-13 12:26:09,029 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:26:09,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 12:26:09,030 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,031 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9 empty. 2023-02-13 12:26:09,031 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,031 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-13 12:26:09,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 12:26:09,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 12:26:09,441 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-13 12:26:09,444 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => c9192eaa901c4d8dc716af4d533a03e9, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:26:09,457 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:09,457 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing c9192eaa901c4d8dc716af4d533a03e9, disabling compactions & flushes 2023-02-13 12:26:09,457 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,457 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,457 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. after waiting 0 ms 2023-02-13 12:26:09,457 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,458 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,458 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for c9192eaa901c4d8dc716af4d533a03e9: Waiting for close lock at 1676291169457Disabling compacts and flushes for region at 1676291169457Disabling writes for close at 1676291169457Writing region close event to WAL at 1676291169458 (+1 ms)Closed at 1676291169458 2023-02-13 12:26:09,461 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:26:09,462 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291169462"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291169462"}]},"ts":"1676291169462"} 2023-02-13 12:26:09,464 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 12:26:09,465 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:26:09,465 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291169465"}]},"ts":"1676291169465"} 2023-02-13 12:26:09,467 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-13 12:26:09,510 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=c9192eaa901c4d8dc716af4d533a03e9, ASSIGN}] 2023-02-13 12:26:09,512 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=c9192eaa901c4d8dc716af4d533a03e9, ASSIGN 2023-02-13 12:26:09,513 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=c9192eaa901c4d8dc716af4d533a03e9, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:26:09,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 12:26:09,665 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c9192eaa901c4d8dc716af4d533a03e9, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:09,665 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291169665"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291169665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291169665"}]},"ts":"1676291169665"} 2023-02-13 12:26:09,668 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure c9192eaa901c4d8dc716af4d533a03e9, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:26:09,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c9192eaa901c4d8dc716af4d533a03e9, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.', STARTKEY => '', ENDKEY => ''} 2023-02-13 12:26:09,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:09,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,831 INFO [StoreOpener-c9192eaa901c4d8dc716af4d533a03e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,833 DEBUG [StoreOpener-c9192eaa901c4d8dc716af4d533a03e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:09,834 DEBUG [StoreOpener-c9192eaa901c4d8dc716af4d533a03e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:09,835 INFO [StoreOpener-c9192eaa901c4d8dc716af4d533a03e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9192eaa901c4d8dc716af4d533a03e9 columnFamilyName family1 2023-02-13 12:26:09,838 INFO [StoreOpener-c9192eaa901c4d8dc716af4d533a03e9-1] regionserver.HStore(310): Store=c9192eaa901c4d8dc716af4d533a03e9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:09,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:09,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:26:09,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened c9192eaa901c4d8dc716af4d533a03e9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68831402, jitterRate=0.025667816400527954}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 12:26:09,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for c9192eaa901c4d8dc716af4d533a03e9: Running coprocessor pre-open hook at 1676291169828Writing region info on filesystem at 1676291169828Initializing all the Stores at 1676291169830 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676291169830Cleaning up temporary data from old regions at 1676291169840 (+10 ms)Cleaning up detritus from prior splits at 1676291169841 (+1 ms)Running coprocessor post-open hooks at 1676291169843 (+2 ms)Region opened successfully at 1676291169843 2023-02-13 12:26:09,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., pid=114, masterSystemTime=1676291169821 2023-02-13 12:26:09,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:09,845 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=c9192eaa901c4d8dc716af4d533a03e9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:09,846 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676291169845"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291169845"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291169845"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291169845"}]},"ts":"1676291169845"} 2023-02-13 12:26:09,848 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 12:26:09,848 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure c9192eaa901c4d8dc716af4d533a03e9, server=jenkins-hbase16.apache.org,38471,1676291011293 in 179 msec 2023-02-13 12:26:09,849 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 12:26:09,849 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=c9192eaa901c4d8dc716af4d533a03e9, ASSIGN in 338 msec 2023-02-13 12:26:09,849 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:26:09,849 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291169849"}]},"ts":"1676291169849"} 2023-02-13 12:26:09,850 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-13 12:26:09,861 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:26:09,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 835 msec 2023-02-13 12:26:10,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-13 12:26:10,137 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-13 12:26:10,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:10,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,162 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:10,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:10,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:10,162 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:10,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,163 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,193 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:10,193 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:10,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:10,194 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,194 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:10,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:10,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,195 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:10,195 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:10,195 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:10,195 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:10,195 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:10,195 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:10,195 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:10,196 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-13 12:26:10,215 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113050c864ab1e1144fbb5c2ca026067f070 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113050c864ab1e1144fbb5c2ca026067f070 2023-02-13 12:26:10,619 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/2196f5b576dd41e1b2ffde086bf6343f 2023-02-13 12:26:10,629 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/2196f5b576dd41e1b2ffde086bf6343f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f 2023-02-13 12:26:10,635 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-13 12:26:10,636 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 440ms, sequenceid=5, compaction requested=false 2023-02-13 12:26:10,636 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:10,636 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:10,636 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:10,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:10,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:10,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:10,637 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:10,669 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,669 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:10,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,670 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:10,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:10,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10cd70be[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:10,674 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:10,674 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,680 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:10,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:10,681 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:10,682 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,682 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:10,689 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:10,689 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,689 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:10,689 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:10,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:10,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:10,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:10,698 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:10,699 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:10,699 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:10,699 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,699 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:10,710 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,710 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,710 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:10,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:10,711 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:10,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:10,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,714 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:10,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:10,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:10,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:10,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,728 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,728 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,727 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:10,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:10,727 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:10,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:10,728 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:10,729 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:10,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:10,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:10,829 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:10,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:10,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:10,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:10,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,848 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:10,848 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:10,848 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:10,848 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:10,849 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,849 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,855 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:10,855 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:10,856 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,856 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:10,857 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,857 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,858 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:10,858 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:10,858 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:10,858 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:10,858 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:10,858 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:10,858 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:10,858 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:10,858 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:10,859 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-13 12:26:10,877 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112814e96088f8e643b5a5eb821c943376cc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814e96088f8e643b5a5eb821c943376cc 2023-02-13 12:26:11,285 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/12539f1823c44fcb948e06492165323c 2023-02-13 12:26:11,295 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/12539f1823c44fcb948e06492165323c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c 2023-02-13 12:26:11,300 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c, entries=1, sequenceid=9, filesize=5.0 K 2023-02-13 12:26:11,300 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 441ms, sequenceid=9, compaction requested=false 2023-02-13 12:26:11,301 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:11,301 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:11,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:11,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:11,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:11,301 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:11,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,314 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:11,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:11,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,314 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:11,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:11,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:11,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:11,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:11,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19b4f002[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:11,317 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:11,318 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,327 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,327 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:11,327 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:11,327 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:11,327 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,327 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:11,338 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:11,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:11,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:11,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:11,339 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:11,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:11,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:11,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:11,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:11,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:11,343 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:11,343 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:11,343 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:11,343 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,343 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:11,352 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:11,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:11,352 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:11,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,353 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:11,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,380 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,380 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:11,380 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:11,380 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:11,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:11,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:11,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:11,380 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:11,380 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,381 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:11,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:11,481 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:11,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:11,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:11,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:11,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,488 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:11,488 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:11,488 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:11,488 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:11,489 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,489 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,502 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,502 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:11,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:11,502 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,502 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:11,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:11,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:11,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:11,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:11,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:11,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:11,504 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:11,504 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:11,504 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:11,504 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:11,925 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c2273fe8fda14d18a2c1e3186897731d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c2273fe8fda14d18a2c1e3186897731d 2023-02-13 12:26:11,933 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/118e63e22f4545ed9ba1d4c21a313b2d 2023-02-13 12:26:11,940 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/118e63e22f4545ed9ba1d4c21a313b2d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d 2023-02-13 12:26:11,943 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d, entries=1, sequenceid=13, filesize=4.9 K 2023-02-13 12:26:11,944 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 440ms, sequenceid=13, compaction requested=true 2023-02-13 12:26:11,944 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:11,944 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:11,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:11,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:11,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:11,944 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:11,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,977 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:11,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:11,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,977 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:11,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:11,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:11,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:11,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:11,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@679da714[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:11,980 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:11,980 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,989 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:11,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:11,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:11,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:11,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:11,997 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:11,997 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:11,997 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:11,997 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:11,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:11,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:11,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:11,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:12,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:12,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:12,005 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:12,006 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:12,006 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:12,006 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,006 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:12,014 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,014 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,014 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:12,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:12,015 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:12,017 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,030 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,030 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,030 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:12,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:12,031 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:12,030 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,031 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:12,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:12,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:12,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:12,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:12,032 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:12,032 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:12,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:12,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:12,132 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:12,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:12,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:12,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:12,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,143 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:12,143 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:12,143 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:12,143 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:12,144 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,144 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,152 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:12,152 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:12,153 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:12,153 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:12,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,155 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:12,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:12,155 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:12,155 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:12,156 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:12,156 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:12,156 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:12,156 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:12,156 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:12,156 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:12,579 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205461ca0e2c9ed412bbc4bc95ba23d0b7b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205461ca0e2c9ed412bbc4bc95ba23d0b7b 2023-02-13 12:26:12,985 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/00832c00ec834703b82999ebda5a3742 2023-02-13 12:26:12,996 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/00832c00ec834703b82999ebda5a3742 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742 2023-02-13 12:26:12,999 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742, entries=1, sequenceid=17, filesize=4.9 K 2023-02-13 12:26:13,000 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 844ms, sequenceid=17, compaction requested=true 2023-02-13 12:26:13,000 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:13,000 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:13,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:13,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:13,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:13,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:13,035 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:13,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:13,035 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,036 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:13,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:13,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:13,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:13,039 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b5caf25[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:13,039 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:13,039 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,047 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,047 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:13,047 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:13,047 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:13,047 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,048 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:13,055 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:13,055 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:13,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:13,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:13,059 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:13,059 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:13,059 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:13,059 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,059 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:13,068 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,068 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:13,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,068 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:13,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:13,069 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,070 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,080 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:13,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:13,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:13,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,081 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,081 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:13,081 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:13,080 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:13,081 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:13,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:13,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:13,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:13,182 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:13,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:13,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:13,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:13,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,188 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:13,188 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:13,189 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:13,189 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:13,189 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,189 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,197 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,197 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:13,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:13,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:13,198 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:13,198 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:13,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:13,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,199 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,199 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:13,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:13,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:13,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:13,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:13,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:13,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:13,200 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:13,200 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:13,200 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:13,617 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289ade99ec6cb5477db86085167c5d58c8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512289ade99ec6cb5477db86085167c5d58c8 2023-02-13 12:26:14,025 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/f059e88a13ca4a4f802b04a83113805c 2023-02-13 12:26:14,037 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/f059e88a13ca4a4f802b04a83113805c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c 2023-02-13 12:26:14,043 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c, entries=1, sequenceid=21, filesize=4.9 K 2023-02-13 12:26:14,044 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 844ms, sequenceid=21, compaction requested=true 2023-02-13 12:26:14,044 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:14,044 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:14,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:14,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:14,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:14,044 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:14,060 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,060 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,061 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,061 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:14,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:14,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@306f7e70[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:14,066 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:14,066 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,077 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,077 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:14,078 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:14,078 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:14,078 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,078 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:14,089 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:14,089 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:14,089 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:14,089 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:14,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:14,098 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:14,098 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:14,098 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:14,098 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,098 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:14,132 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,132 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,132 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:14,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:14,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:14,134 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,138 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:14,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:14,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,157 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,157 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:14,157 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:14,157 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:14,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:14,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:14,156 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:14,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:14,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:14,257 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:14,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:14,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:14,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:14,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,273 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:14,273 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:14,273 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:14,273 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:14,274 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,274 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,283 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,283 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:14,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:14,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:14,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,284 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:14,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:14,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:14,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,287 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:14,288 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:14,288 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:14,288 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:14,288 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:14,288 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:14,288 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:14,289 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:14,713 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512313196c1890292428b94b31134b8548635 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512313196c1890292428b94b31134b8548635 2023-02-13 12:26:14,719 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/501471173094424e875feb7eb67a6c6c 2023-02-13 12:26:14,723 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/501471173094424e875feb7eb67a6c6c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c 2023-02-13 12:26:14,728 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c, entries=1, sequenceid=25, filesize=4.9 K 2023-02-13 12:26:14,729 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 441ms, sequenceid=25, compaction requested=true 2023-02-13 12:26:14,729 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:14,729 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:14,729 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:14,729 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:14,729 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,729 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:14,729 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:14,775 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,775 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,775 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,775 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:14,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:14,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ee5bc9f[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:14,777 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:14,777 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,783 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,783 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:14,783 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:14,783 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,784 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:14,783 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:14,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:14,791 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:14,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,791 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:14,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:14,793 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:14,793 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:14,793 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:14,793 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,793 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:14,800 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,800 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,800 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:14,800 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:14,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:14,801 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:14,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,808 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:14,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:14,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:14,808 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:14,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:14,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:14,809 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:14,809 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:14,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:14,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:14,909 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:14,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:14,913 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:14,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:14,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,914 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:14,914 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:14,914 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:14,915 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:14,915 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,915 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,925 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:14,925 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:14,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:14,925 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:14,925 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,926 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:14,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:14,926 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:14,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:14,927 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:14,927 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:14,927 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:15,338 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010120df3f66ce6d4f938b7110fe5bb3510b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010120df3f66ce6d4f938b7110fe5bb3510b 2023-02-13 12:26:15,377 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:26:15,744 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/4560901699ee4c3ab7125a63a14a8c11 2023-02-13 12:26:15,756 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/4560901699ee4c3ab7125a63a14a8c11 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11 2023-02-13 12:26:15,760 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11, entries=1, sequenceid=29, filesize=4.9 K 2023-02-13 12:26:15,761 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 834ms, sequenceid=29, compaction requested=true 2023-02-13 12:26:15,761 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:15,761 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:15,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:15,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:15,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:15,761 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:15,800 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,800 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:15,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:15,801 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,801 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:15,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:15,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:15,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:15,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:15,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ca1f50[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:15,806 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:15,806 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,817 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:15,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:15,817 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:15,818 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,818 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:15,825 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:15,825 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,825 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:15,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:15,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:15,825 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:15,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:15,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:15,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:15,835 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:15,835 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:15,835 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:15,835 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:15,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,836 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:15,847 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,847 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,847 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:15,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:15,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:15,849 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:15,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,852 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,864 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:15,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:15,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:15,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:15,865 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:15,864 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:15,865 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:15,865 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:15,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:15,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:15,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:15,965 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:15,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:15,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:15,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:15,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,978 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:15,979 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:15,979 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:15,979 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:15,979 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,979 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,989 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:15,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:15,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:15,990 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:15,990 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:15,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:15,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:15,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:15,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:15,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:15,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:15,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:15,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:15,992 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:15,992 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:15,992 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:16,410 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bc54336c91e8441f8f12a79c23120ef2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc54336c91e8441f8f12a79c23120ef2 2023-02-13 12:26:16,817 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/f57d4a59764d4346a181a0e5107d9d1d 2023-02-13 12:26:16,828 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/f57d4a59764d4346a181a0e5107d9d1d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d 2023-02-13 12:26:16,832 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d, entries=1, sequenceid=33, filesize=4.9 K 2023-02-13 12:26:16,833 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 841ms, sequenceid=33, compaction requested=true 2023-02-13 12:26:16,833 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:16,833 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:16,833 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:16,833 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:16,833 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,833 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:16,833 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:16,875 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,875 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:16,875 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,875 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:16,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:16,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:16,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ac1ad6b[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:16,878 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:16,879 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,889 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:16,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:16,889 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:16,889 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,889 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:16,926 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:16,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:16,927 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:16,927 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:16,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:16,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:16,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:16,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:16,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:16,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:16,934 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:16,934 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:16,934 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:16,934 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,934 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:16,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:16,950 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:16,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:16,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:16,952 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,955 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:16,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,966 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,966 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:16,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:16,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:16,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:16,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,967 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,968 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:16,967 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:16,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:16,968 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:17,068 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:17,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:17,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:17,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:17,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,082 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:17,082 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:17,082 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:17,083 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:17,083 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,083 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,091 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:17,092 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:17,092 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,092 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:17,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:17,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,094 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:17,094 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,094 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:17,094 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:17,094 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:17,095 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:17,095 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:17,095 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:17,095 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:17,095 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:17,516 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090765e28e9708084ae4be5504886187da0c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015090765e28e9708084ae4be5504886187da0c 2023-02-13 12:26:17,923 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/1b11a2e3f69c4d5c9be27c1283152c79 2023-02-13 12:26:17,935 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/1b11a2e3f69c4d5c9be27c1283152c79 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79 2023-02-13 12:26:17,939 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79, entries=1, sequenceid=37, filesize=4.9 K 2023-02-13 12:26:17,940 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 845ms, sequenceid=37, compaction requested=true 2023-02-13 12:26:17,940 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:17,940 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:17,940 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:17,940 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:17,940 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,940 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:17,940 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:17,975 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,975 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:17,975 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,976 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:17,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:17,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:17,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:17,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:17,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6196dabd[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:17,980 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:17,980 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,989 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:17,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:17,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:17,989 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:17,990 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:17,990 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:18,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:18,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:18,000 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:18,000 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:18,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:18,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:18,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:18,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:18,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:18,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:18,010 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:18,010 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:18,010 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:18,010 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,010 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:18,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:18,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:18,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,022 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:18,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:18,024 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:18,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:18,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:18,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,027 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:18,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:18,039 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:18,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:18,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:18,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:18,039 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,040 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,040 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:18,040 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:18,040 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:18,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:18,041 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,141 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:18,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:18,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:18,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:18,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,155 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:18,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:18,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-13 12:26:18,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-13 12:26:18,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,156 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,167 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:18,167 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:18,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:18,168 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:18,168 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:18,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-13 12:26:18,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:18,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-13 12:26:18,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:18,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-13 12:26:18,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 12:26:18,171 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:18,171 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:18,171 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. started... 2023-02-13 12:26:18,171 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing c9192eaa901c4d8dc716af4d533a03e9 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-13 12:26:18,589 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112020ee910d15c9445aa682bd4a367a605e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112020ee910d15c9445aa682bd4a367a605e 2023-02-13 12:26:18,997 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/45f61a1383bf4612be8fbba1d21a8c04 2023-02-13 12:26:19,008 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/45f61a1383bf4612be8fbba1d21a8c04 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04 2023-02-13 12:26:19,013 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04, entries=1, sequenceid=41, filesize=4.9 K 2023-02-13 12:26:19,014 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9192eaa901c4d8dc716af4d533a03e9 in 843ms, sequenceid=41, compaction requested=true 2023-02-13 12:26:19,014 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:19,014 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:19,014 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 12:26:19,015 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 12:26:19,015 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,015 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-13 12:26:19,015 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-13 12:26:19,127 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,127 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:19,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:19,129 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,129 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:19,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:19,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:19,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:19,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-13 12:26:19,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42b49354[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:19,135 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-13 12:26:19,135 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,185 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,186 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-13 12:26:19,186 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-13 12:26:19,186 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:19,187 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,187 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:19,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-13 12:26:19,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:19,200 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,200 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-13 12:26:19,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:19,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:19,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:19,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:19,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:19,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:19,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-13 12:26:19,212 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-13 12:26:19,212 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:19,212 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:19,212 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,213 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:19,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,225 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:19,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:19,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:19,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:19,226 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:19,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:19,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:19,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,229 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:19,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:19,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:19,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-13 12:26:19,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:19,242 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:19,242 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:19,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:19,243 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:19,243 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:19,243 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,244 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,344 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:19,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:19,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-13 12:26:19,361 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:26:19,362 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:26:19,363 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:26:19,363 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291779363 2023-02-13 12:26:19,363 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-13 12:26:19,364 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-13 12:26:19,364 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:19,364 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x464c4440 to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:26:19,375 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7acfbafd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:26:19,376 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:19,816 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cf06cfdb47514560b4fe63ffbe91f231 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103cf06cfdb47514560b4fe63ffbe91f231 2023-02-13 12:26:19,818 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:19,832 DEBUG [MobCompactor-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:26:19,834 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:58086, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:26:19,840 DEBUG [MobCompactor-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:26:19,841 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:49826, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:26:19,848 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbf273aac39442c28ad63bc105ccacce first=Optional[r5] last=Optional[r6] 2023-02-13 12:26:19,849 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbf273aac39442c28ad63bc105ccacce}] 2023-02-13 12:26:19,855 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbf273aac39442c28ad63bc105ccacce for inclusion in c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:19,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-13 12:26:19,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:26:19,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:19,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cbf273aac39442c28ad63bc105ccacce to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__1pflnc72gtbsdsuvf1eqohcqk7okv73oca76916fhgr8fg9cacia33g71fph5bhj/family1/cbf273aac39442c28ad63bc105ccacce 2023-02-13 12:26:19,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__1pflnc72gtbsdsuvf1eqohcqk7okv73oca76916fhgr8fg9cacia33g71fph5bhj/family1/cbf273aac39442c28ad63bc105ccacce as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_ 2023-02-13 12:26:19,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__1pflnc72gtbsdsuvf1eqohcqk7okv73oca76916fhgr8fg9cacia33g71fph5bhj/family1/cbf273aac39442c28ad63bc105ccacce into c9192eaa901c4d8dc716af4d533a03e9/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_ - updating store file list. 2023-02-13 12:26:19,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_ into c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:19,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__1pflnc72gtbsdsuvf1eqohcqk7okv73oca76916fhgr8fg9cacia33g71fph5bhj/family1/cbf273aac39442c28ad63bc105ccacce into c9192eaa901c4d8dc716af4d533a03e9/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_) 2023-02-13 12:26:19,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__1pflnc72gtbsdsuvf1eqohcqk7okv73oca76916fhgr8fg9cacia33g71fph5bhj/family1/cbf273aac39442c28ad63bc105ccacce 2023-02-13 12:26:19,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-13 12:26:19,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-13 12:26:19,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(1912): c9192eaa901c4d8dc716af4d533a03e9/family1 is initiating minor compaction 2023-02-13 12:26:19,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6de46077; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:26:19,867 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c9192eaa901c4d8dc716af4d533a03e9/family1 in testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:19,867 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp, totalSize=49.7 K 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 12539f1823c44fcb948e06492165323c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 118e63e22f4545ed9ba1d4c21a313b2d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 00832c00ec834703b82999ebda5a3742, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting f059e88a13ca4a4f802b04a83113805c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 501471173094424e875feb7eb67a6c6c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 4560901699ee4c3ab7125a63a14a8c11, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting f57d4a59764d4346a181a0e5107d9d1d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 1b11a2e3f69c4d5c9be27c1283152c79, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 45f61a1383bf4612be8fbba1d21a8c04, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-13 12:26:19,868 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 73fee194503c4aa9800d5e82a658f828_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-13 12:26:19,869 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:19,870 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:19,872 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010120df3f66ce6d4f938b7110fe5bb3510b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010120df3f66ce6d4f938b7110fe5bb3510b 2023-02-13 12:26:19,873 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc54336c91e8441f8f12a79c23120ef2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103bc54336c91e8441f8f12a79c23120ef2 2023-02-13 12:26:19,873 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-13 12:26:19,873 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-13 12:26:19,874 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:19,874 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:19,893 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9192eaa901c4d8dc716af4d533a03e9#family1#compaction#166 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:26:20,310 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/b7a58a39218e4a6b98c0cc4674459095 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/b7a58a39218e4a6b98c0cc4674459095 2023-02-13 12:26:20,312 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c637aa78ecec48909e9eb0a4d8b16adb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231c637aa78ecec48909e9eb0a4d8b16adb 2023-02-13 12:26:20,312 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:20,315 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c9192eaa901c4d8dc716af4d533a03e9/family1 of c9192eaa901c4d8dc716af4d533a03e9 into b7a58a39218e4a6b98c0cc4674459095(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:26:20,315 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:20,315 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676291179867; duration=0sec 2023-02-13 12:26:20,315 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:26:20,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_] to archive 2023-02-13 12:26:20,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:20,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/12539f1823c44fcb948e06492165323c 2023-02-13 12:26:20,324 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/118e63e22f4545ed9ba1d4c21a313b2d 2023-02-13 12:26:20,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/00832c00ec834703b82999ebda5a3742 2023-02-13 12:26:20,328 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f059e88a13ca4a4f802b04a83113805c 2023-02-13 12:26:20,329 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/501471173094424e875feb7eb67a6c6c 2023-02-13 12:26:20,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/4560901699ee4c3ab7125a63a14a8c11 2023-02-13 12:26:20,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f57d4a59764d4346a181a0e5107d9d1d 2023-02-13 12:26:20,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/1b11a2e3f69c4d5c9be27c1283152c79 2023-02-13 12:26:20,333 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/45f61a1383bf4612be8fbba1d21a8c04 2023-02-13 12:26:20,334 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/73fee194503c4aa9800d5e82a658f828_SeqId_44_ 2023-02-13 12:26:20,339 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/eb894f389b21489bb11165f67d04565d first=Optional[r1] last=Optional[r4] 2023-02-13 12:26:20,340 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/eb894f389b21489bb11165f67d04565d}] 2023-02-13 12:26:20,341 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/eb894f389b21489bb11165f67d04565d for inclusion in c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:20,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-13 12:26:20,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:26:20,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:20,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/eb894f389b21489bb11165f67d04565d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hpu5oe6gddopveb3a8i71gk2mr5a2fh1vgbhpc0t5t5vkmfs9pt64nmp20mmmobf/family1/eb894f389b21489bb11165f67d04565d 2023-02-13 12:26:20,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hpu5oe6gddopveb3a8i71gk2mr5a2fh1vgbhpc0t5t5vkmfs9pt64nmp20mmmobf/family1/eb894f389b21489bb11165f67d04565d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_ 2023-02-13 12:26:20,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hpu5oe6gddopveb3a8i71gk2mr5a2fh1vgbhpc0t5t5vkmfs9pt64nmp20mmmobf/family1/eb894f389b21489bb11165f67d04565d into c9192eaa901c4d8dc716af4d533a03e9/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_ - updating store file list. 2023-02-13 12:26:20,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_ into c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:20,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hpu5oe6gddopveb3a8i71gk2mr5a2fh1vgbhpc0t5t5vkmfs9pt64nmp20mmmobf/family1/eb894f389b21489bb11165f67d04565d into c9192eaa901c4d8dc716af4d533a03e9/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_) 2023-02-13 12:26:20,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__hpu5oe6gddopveb3a8i71gk2mr5a2fh1vgbhpc0t5t5vkmfs9pt64nmp20mmmobf/family1/eb894f389b21489bb11165f67d04565d 2023-02-13 12:26:20,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-13 12:26:20,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-13 12:26:20,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(1912): c9192eaa901c4d8dc716af4d533a03e9/family1 is initiating minor compaction (all files) 2023-02-13 12:26:20,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@476842e4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:26:20,351 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c9192eaa901c4d8dc716af4d533a03e9/family1 in testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:20,352 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/b7a58a39218e4a6b98c0cc4674459095, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp, totalSize=178.1 K 2023-02-13 12:26:20,352 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting 2196f5b576dd41e1b2ffde086bf6343f, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836144 2023-02-13 12:26:20,352 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting b7a58a39218e4a6b98c0cc4674459095, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236144 2023-02-13 12:26:20,353 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] compactions.Compactor(207): Compacting e3695f285c0040dead5e8d97834a4617_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965236144 2023-02-13 12:26:20,353 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:20,354 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:20,355 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c2273fe8fda14d18a2c1e3186897731d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201c2273fe8fda14d18a2c1e3186897731d 2023-02-13 12:26:20,355 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205461ca0e2c9ed412bbc4bc95ba23d0b7b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205461ca0e2c9ed412bbc4bc95ba23d0b7b 2023-02-13 12:26:20,356 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512289ade99ec6cb5477db86085167c5d58c8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512289ade99ec6cb5477db86085167c5d58c8 2023-02-13 12:26:20,357 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512313196c1890292428b94b31134b8548635 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512313196c1890292428b94b31134b8548635 2023-02-13 12:26:20,357 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-13 12:26:20,357 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-13 12:26:20,358 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:20,358 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:20,364 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9192eaa901c4d8dc716af4d533a03e9#family1#compaction#167 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-13 12:26:20,807 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c1931d4685f44c518501841352501487 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130c1931d4685f44c518501841352501487 2023-02-13 12:26:20,808 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/.tmp/family1/2afda6e4e83b4cf69929a191e6bdb5d8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2afda6e4e83b4cf69929a191e6bdb5d8 2023-02-13 12:26:20,808 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:20,812 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c9192eaa901c4d8dc716af4d533a03e9/family1 of c9192eaa901c4d8dc716af4d533a03e9 into 2afda6e4e83b4cf69929a191e6bdb5d8(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:26:20,812 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:20,812 INFO [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676291180351; duration=0sec 2023-02-13 12:26:20,813 DEBUG [RS:0;jenkins-hbase16:38471-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 12:26:20,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/b7a58a39218e4a6b98c0cc4674459095, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_] to archive 2023-02-13 12:26:20,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:20,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/2196f5b576dd41e1b2ffde086bf6343f 2023-02-13 12:26:20,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/b7a58a39218e4a6b98c0cc4674459095 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/b7a58a39218e4a6b98c0cc4674459095 2023-02-13 12:26:20,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_ to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/e3695f285c0040dead5e8d97834a4617_SeqId_47_ 2023-02-13 12:26:20,831 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7dd912adebd341799358e5251e89ee9c first=Optional[r00] last=Optional[r8] 2023-02-13 12:26:20,832 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7dd912adebd341799358e5251e89ee9c}] 2023-02-13 12:26:20,834 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7dd912adebd341799358e5251e89ee9c for inclusion in c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:20,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-13 12:26:20,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last= 2023-02-13 12:26:20,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for c9192eaa901c4d8dc716af4d533a03e9: 2023-02-13 12:26:20,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7dd912adebd341799358e5251e89ee9c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gktt4di4k5pjk598b8ol5ih2egl3ai4hnuf89q82te4lcj1mjsv5ovbfs13bn7nc/family1/7dd912adebd341799358e5251e89ee9c 2023-02-13 12:26:20,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gktt4di4k5pjk598b8ol5ih2egl3ai4hnuf89q82te4lcj1mjsv5ovbfs13bn7nc/family1/7dd912adebd341799358e5251e89ee9c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f3d2de09b21547739c578a950814d48c_SeqId_50_ 2023-02-13 12:26:20,839 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gktt4di4k5pjk598b8ol5ih2egl3ai4hnuf89q82te4lcj1mjsv5ovbfs13bn7nc/family1/7dd912adebd341799358e5251e89ee9c into c9192eaa901c4d8dc716af4d533a03e9/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f3d2de09b21547739c578a950814d48c_SeqId_50_ - updating store file list. 2023-02-13 12:26:20,842 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f3d2de09b21547739c578a950814d48c_SeqId_50_ into c9192eaa901c4d8dc716af4d533a03e9/family1 2023-02-13 12:26:20,842 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gktt4di4k5pjk598b8ol5ih2egl3ai4hnuf89q82te4lcj1mjsv5ovbfs13bn7nc/family1/7dd912adebd341799358e5251e89ee9c into c9192eaa901c4d8dc716af4d533a03e9/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/family1/f3d2de09b21547739c578a950814d48c_SeqId_50_) 2023-02-13 12:26:20,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMinorCompactionWithMonthlyPolicy__gktt4di4k5pjk598b8ol5ih2egl3ai4hnuf89q82te4lcj1mjsv5ovbfs13bn7nc/family1/7dd912adebd341799358e5251e89ee9c 2023-02-13 12:26:20,845 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:20,845 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:20,846 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112020ee910d15c9445aa682bd4a367a605e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112020ee910d15c9445aa682bd4a367a605e 2023-02-13 12:26:20,847 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814e96088f8e643b5a5eb821c943376cc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112814e96088f8e643b5a5eb821c943376cc 2023-02-13 12:26:20,847 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113050c864ab1e1144fbb5c2ca026067f070 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113050c864ab1e1144fbb5c2ca026067f070 2023-02-13 12:26:20,847 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-13 12:26:20,848 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-13 12:26:20,848 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:20,848 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:20,848 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-13 12:26:20,848 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-13 12:26:20,848 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:26:20,850 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-13 12:26:20,851 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.4890 sec 2023-02-13 12:26:20,957 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=253 (was 246) - Thread LEAK? -, OpenFileDescriptor=644 (was 613) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=44 (was 47), ProcessCount=166 (was 167), AvailableMemoryMB=5866 (was 5872) 2023-02-13 12:26:20,967 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=253, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=44, ProcessCount=166, AvailableMemoryMB=5866 2023-02-13 12:26:20,982 DEBUG [Listener at localhost.localdomain/33629] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-13 12:26:21,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.HMaster$4(2102): Client=jenkins//188.40.62.62 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-13 12:26:21,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-13 12:26:21,025 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 12:26:21,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(697): Client=jenkins//188.40.62.62 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-13 12:26:21,025 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 12:26:21,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac empty. 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd empty. 2023-02-13 12:26:21,027 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba empty. 2023-02-13 12:26:21,028 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:21,028 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:21,028 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba 2023-02-13 12:26:21,028 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-13 12:26:21,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:21,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:21,437 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-13 12:26:21,440 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4b661928fda9be3aac0c0235707673ac, NAME => 'testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:26:21,441 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 070d047471747182c749aaedda740bba, NAME => 'testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:26:21,442 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => adac9b9ed7bc6febc3aa79d4b3cce9dd, NAME => 'testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/.tmp 2023-02-13 12:26:21,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:21,851 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:21,851 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 070d047471747182c749aaedda740bba, disabling compactions & flushes 2023-02-13 12:26:21,852 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. after waiting 0 ms 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:21,852 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 4b661928fda9be3aac0c0235707673ac, disabling compactions & flushes 2023-02-13 12:26:21,853 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:21,853 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:21,853 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. after waiting 0 ms 2023-02-13 12:26:21,852 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 070d047471747182c749aaedda740bba: Waiting for close lock at 1676291181851Disabling compacts and flushes for region at 1676291181851Disabling writes for close at 1676291181852 (+1 ms)Writing region close event to WAL at 1676291181852Closed at 1676291181852 2023-02-13 12:26:21,853 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:21,853 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:21,853 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 4b661928fda9be3aac0c0235707673ac: Waiting for close lock at 1676291181852Disabling compacts and flushes for region at 1676291181852Disabling writes for close at 1676291181853 (+1 ms)Writing region close event to WAL at 1676291181853Closed at 1676291181853 2023-02-13 12:26:21,853 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:21,854 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing adac9b9ed7bc6febc3aa79d4b3cce9dd, disabling compactions & flushes 2023-02-13 12:26:21,854 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:21,854 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:21,854 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. after waiting 0 ms 2023-02-13 12:26:21,854 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:21,854 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:21,854 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: Waiting for close lock at 1676291181854Disabling compacts and flushes for region at 1676291181854Disabling writes for close at 1676291181854Writing region close event to WAL at 1676291181854Closed at 1676291181854 2023-02-13 12:26:21,859 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 12:26:21,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291181861"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291181861"}]},"ts":"1676291181861"} 2023-02-13 12:26:21,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291181861"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291181861"}]},"ts":"1676291181861"} 2023-02-13 12:26:21,861 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676291181861"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676291181861"}]},"ts":"1676291181861"} 2023-02-13 12:26:21,864 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-13 12:26:21,866 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 12:26:21,866 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291181866"}]},"ts":"1676291181866"} 2023-02-13 12:26:21,867 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-13 12:26:22,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4b661928fda9be3aac0c0235707673ac, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=adac9b9ed7bc6febc3aa79d4b3cce9dd, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=070d047471747182c749aaedda740bba, ASSIGN}] 2023-02-13 12:26:22,132 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=adac9b9ed7bc6febc3aa79d4b3cce9dd, ASSIGN 2023-02-13 12:26:22,133 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=070d047471747182c749aaedda740bba, ASSIGN 2023-02-13 12:26:22,133 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4b661928fda9be3aac0c0235707673ac, ASSIGN 2023-02-13 12:26:22,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:22,134 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=adac9b9ed7bc6febc3aa79d4b3cce9dd, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:26:22,134 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=070d047471747182c749aaedda740bba, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:26:22,134 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4b661928fda9be3aac0c0235707673ac, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,38471,1676291011293; forceNewPlan=false, retain=false 2023-02-13 12:26:22,287 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=4b661928fda9be3aac0c0235707673ac, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,287 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=070d047471747182c749aaedda740bba, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,288 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291182287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291182287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291182287"}]},"ts":"1676291182287"} 2023-02-13 12:26:22,288 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291182287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291182287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291182287"}]},"ts":"1676291182287"} 2023-02-13 12:26:22,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=adac9b9ed7bc6febc3aa79d4b3cce9dd, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676291182287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676291182287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676291182287"}]},"ts":"1676291182287"} 2023-02-13 12:26:22,290 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 4b661928fda9be3aac0c0235707673ac, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:26:22,292 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure 070d047471747182c749aaedda740bba, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:26:22,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure adac9b9ed7bc6febc3aa79d4b3cce9dd, server=jenkins-hbase16.apache.org,38471,1676291011293}] 2023-02-13 12:26:22,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:22,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => adac9b9ed7bc6febc3aa79d4b3cce9dd, NAME => 'testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.', STARTKEY => '1', ENDKEY => '2'} 2023-02-13 12:26:22,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:22,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,473 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,475 DEBUG [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 2023-02-13 12:26:22,475 DEBUG [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 2023-02-13 12:26:22,475 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region adac9b9ed7bc6febc3aa79d4b3cce9dd columnFamilyName family1 2023-02-13 12:26:22,476 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] regionserver.HStore(310): Store=adac9b9ed7bc6febc3aa79d4b3cce9dd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,477 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,478 DEBUG [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2 2023-02-13 12:26:22,478 DEBUG [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2 2023-02-13 12:26:22,478 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region adac9b9ed7bc6febc3aa79d4b3cce9dd columnFamilyName family2 2023-02-13 12:26:22,479 INFO [StoreOpener-adac9b9ed7bc6febc3aa79d4b3cce9dd-1] regionserver.HStore(310): Store=adac9b9ed7bc6febc3aa79d4b3cce9dd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:26:22,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:22,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:26:22,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened adac9b9ed7bc6febc3aa79d4b3cce9dd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66628418, jitterRate=-0.007159203290939331}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:26:22,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: Running coprocessor pre-open hook at 1676291182448Writing region info on filesystem at 1676291182448Initializing all the Stores at 1676291182450 (+2 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182450Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182450Cleaning up temporary data from old regions at 1676291182481 (+31 ms)Cleaning up detritus from prior splits at 1676291182481Running coprocessor post-open hooks at 1676291182483 (+2 ms)Region opened successfully at 1676291182483 2023-02-13 12:26:22,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd., pid=122, masterSystemTime=1676291182443 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:22,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:22,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4b661928fda9be3aac0c0235707673ac, NAME => 'testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.', STARTKEY => '', ENDKEY => '1'} 2023-02-13 12:26:22,485 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=adac9b9ed7bc6febc3aa79d4b3cce9dd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:22,485 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676291182485"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291182485"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291182485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291182485"}]},"ts":"1676291182485"} 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,487 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-02-13 12:26:22,487 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure adac9b9ed7bc6febc3aa79d4b3cce9dd, server=jenkins-hbase16.apache.org,38471,1676291011293 in 194 msec 2023-02-13 12:26:22,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=adac9b9ed7bc6febc3aa79d4b3cce9dd, ASSIGN in 359 msec 2023-02-13 12:26:22,504 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,505 DEBUG [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1 2023-02-13 12:26:22,505 DEBUG [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1 2023-02-13 12:26:22,505 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b661928fda9be3aac0c0235707673ac columnFamilyName family1 2023-02-13 12:26:22,506 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] regionserver.HStore(310): Store=4b661928fda9be3aac0c0235707673ac/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,506 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,507 DEBUG [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2 2023-02-13 12:26:22,507 DEBUG [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2 2023-02-13 12:26:22,508 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b661928fda9be3aac0c0235707673ac columnFamilyName family2 2023-02-13 12:26:22,508 INFO [StoreOpener-4b661928fda9be3aac0c0235707673ac-1] regionserver.HStore(310): Store=4b661928fda9be3aac0c0235707673ac/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:26:22,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:22,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:26:22,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 4b661928fda9be3aac0c0235707673ac; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66737173, jitterRate=-0.005538627505302429}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:26:22,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 4b661928fda9be3aac0c0235707673ac: Running coprocessor pre-open hook at 1676291182485Writing region info on filesystem at 1676291182485Initializing all the Stores at 1676291182486 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182486Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182486Cleaning up temporary data from old regions at 1676291182510 (+24 ms)Cleaning up detritus from prior splits at 1676291182510Running coprocessor post-open hooks at 1676291182513 (+3 ms)Region opened successfully at 1676291182513 2023-02-13 12:26:22,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac., pid=120, masterSystemTime=1676291182443 2023-02-13 12:26:22,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:22,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:22,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:22,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 070d047471747182c749aaedda740bba, NAME => 'testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.', STARTKEY => '2', ENDKEY => ''} 2023-02-13 12:26:22,515 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=4b661928fda9be3aac0c0235707673ac, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 12:26:22,515 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291182515"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291182515"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291182515"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291182515"}]},"ts":"1676291182515"} 2023-02-13 12:26:22,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,518 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-02-13 12:26:22,518 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 4b661928fda9be3aac0c0235707673ac, server=jenkins-hbase16.apache.org,38471,1676291011293 in 226 msec 2023-02-13 12:26:22,519 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=4b661928fda9be3aac0c0235707673ac, ASSIGN in 390 msec 2023-02-13 12:26:22,538 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,539 DEBUG [StoreOpener-070d047471747182c749aaedda740bba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1 2023-02-13 12:26:22,539 DEBUG [StoreOpener-070d047471747182c749aaedda740bba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1 2023-02-13 12:26:22,540 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 070d047471747182c749aaedda740bba columnFamilyName family1 2023-02-13 12:26:22,541 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] regionserver.HStore(310): Store=070d047471747182c749aaedda740bba/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,541 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,542 DEBUG [StoreOpener-070d047471747182c749aaedda740bba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2 2023-02-13 12:26:22,542 DEBUG [StoreOpener-070d047471747182c749aaedda740bba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2 2023-02-13 12:26:22,542 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 070d047471747182c749aaedda740bba columnFamilyName family2 2023-02-13 12:26:22,543 INFO [StoreOpener-070d047471747182c749aaedda740bba-1] regionserver.HStore(310): Store=070d047471747182c749aaedda740bba/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 12:26:22,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba 2023-02-13 12:26:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba 2023-02-13 12:26:22,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 12:26:22,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 070d047471747182c749aaedda740bba 2023-02-13 12:26:22,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 12:26:22,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 070d047471747182c749aaedda740bba; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74327837, jitterRate=0.107571080327034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 12:26:22,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 070d047471747182c749aaedda740bba: Running coprocessor pre-open hook at 1676291182515Writing region info on filesystem at 1676291182515Initializing all the Stores at 1676291182516 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182516Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676291182516Cleaning up temporary data from old regions at 1676291182544 (+28 ms)Cleaning up detritus from prior splits at 1676291182544Running coprocessor post-open hooks at 1676291182547 (+3 ms)Region opened successfully at 1676291182547 2023-02-13 12:26:22,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba., pid=121, masterSystemTime=1676291182443 2023-02-13 12:26:22,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:22,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:22,549 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=070d047471747182c749aaedda740bba, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:22,549 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676291182549"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676291182549"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676291182549"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676291182549"}]},"ts":"1676291182549"} 2023-02-13 12:26:22,551 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-13 12:26:22,551 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure 070d047471747182c749aaedda740bba, server=jenkins-hbase16.apache.org,38471,1676291011293 in 258 msec 2023-02-13 12:26:22,552 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-02-13 12:26:22,553 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=070d047471747182c749aaedda740bba, ASSIGN in 423 msec 2023-02-13 12:26:22,553 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 12:26:22,553 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676291182553"}]},"ts":"1676291182553"} 2023-02-13 12:26:22,554 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-13 12:26:22,567 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 12:26:22,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.5440 sec 2023-02-13 12:26:23,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-13 12:26:23,137 INFO [Listener at localhost.localdomain/33629] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-13 12:26:23,149 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:23,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:23,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:23,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:23,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 12:26:23,157 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 12:26:23,157 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:23,157 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 12:26:23,157 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 12:26:23,157 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:23,157 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:23,191 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:23,191 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:23,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:23,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:23,192 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:23,192 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:23,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:23,193 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 12:26:23,194 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 12:26:23,194 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 12:26:23,194 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:23,194 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 12:26:23,194 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:23,194 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. started... 2023-02-13 12:26:23,194 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:26:23,194 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:23,194 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:23,195 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing adac9b9ed7bc6febc3aa79d4b3cce9dd 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:26:23,194 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:23,196 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. started... 2023-02-13 12:26:23,195 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. started... 2023-02-13 12:26:23,197 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4b661928fda9be3aac0c0235707673ac 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:26:23,197 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 070d047471747182c749aaedda740bba 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-13 12:26:23,244 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302133abb0d81b4d04aaa8ed5a1420bd75262 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302133abb0d81b4d04aaa8ed5a1420bd75262 2023-02-13 12:26:23,648 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213bcb77e0dcafa42c58a0b35abaa7fc1eb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213bcb77e0dcafa42c58a0b35abaa7fc1eb 2023-02-13 12:26:23,648 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213de1acb571ece4c3ea19db6ecf58dc326 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213de1acb571ece4c3ea19db6ecf58dc326 2023-02-13 12:26:23,656 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5 2023-02-13 12:26:23,665 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/56e050ca9de34f609611048c278b3ed0 2023-02-13 12:26:24,062 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/f36fa54339184154b1d39c2a0b5b13b4 2023-02-13 12:26:24,092 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213b6bd8b6b53224f6b90adfebb9ca51828 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213b6bd8b6b53224f6b90adfebb9ca51828 2023-02-13 12:26:24,098 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213128d1d63003a40e3bc55b3c00ef2a706 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213128d1d63003a40e3bc55b3c00ef2a706 2023-02-13 12:26:24,497 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967 2023-02-13 12:26:24,503 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/8314e500fcb74be7a31916859f48fca2 2023-02-13 12:26:24,508 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5 2023-02-13 12:26:24,509 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213f499863241a545318641d80dbb580ced to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213f499863241a545318641d80dbb580ced 2023-02-13 12:26:24,509 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/56e050ca9de34f609611048c278b3ed0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0 2023-02-13 12:26:24,531 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 12:26:24,532 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/8314e500fcb74be7a31916859f48fca2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2 2023-02-13 12:26:24,532 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 12:26:24,533 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967 2023-02-13 12:26:24,535 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:26:24,536 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for adac9b9ed7bc6febc3aa79d4b3cce9dd in 1341ms, sequenceid=5, compaction requested=false 2023-02-13 12:26:24,536 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-13 12:26:24,536 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:26:24,536 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:24,536 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:24,536 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:24,536 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 070d047471747182c749aaedda740bba in 1339ms, sequenceid=5, compaction requested=false 2023-02-13 12:26:24,537 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:24,537 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:24,537 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:24,919 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/72f2d27cf2034d63b637e41ec3478325 2023-02-13 12:26:24,930 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/f36fa54339184154b1d39c2a0b5b13b4 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4 2023-02-13 12:26:24,955 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4, entries=4, sequenceid=5, filesize=5.5 K 2023-02-13 12:26:24,955 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/72f2d27cf2034d63b637e41ec3478325 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325 2023-02-13 12:26:24,959 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325, entries=2, sequenceid=5, filesize=5.1 K 2023-02-13 12:26:24,960 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 4b661928fda9be3aac0c0235707673ac in 1764ms, sequenceid=5, compaction requested=false 2023-02-13 12:26:24,960 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:24,960 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:24,960 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:24,960 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:24,960 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:24,960 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 12:26:24,960 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 12:26:24,991 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:24,991 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:24,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:24,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:24,992 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:24,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:24,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:24,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:24,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:24,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:24,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 12:26:24,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@165034cd[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:24,996 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 12:26:24,996 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,005 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,006 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 12:26:25,006 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 12:26:25,006 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:25,006 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,006 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:25,016 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 12:26:25,016 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:25,017 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 12:26:25,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:25,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:25,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:25,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:25,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:25,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:25,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:25,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:25,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 12:26:25,025 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 12:26:25,026 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:25,026 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:25,026 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 12:26:25,026 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:25,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:25,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:25,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,033 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:25,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:25,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:25,034 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:25,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:25,037 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:25,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:25,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,050 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:25,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:25,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 12:26:25,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:25,050 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:25,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:25,051 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:25,051 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:25,051 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:25,051 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:25,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:25,152 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:25,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:25,158 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:25,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:25,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:25,165 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:25,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:25,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 12:26:25,167 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 12:26:25,167 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:25,167 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 12:26:25,167 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 12:26:25,168 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,168 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:25,174 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,174 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:25,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:25,175 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:25,175 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:25,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:25,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:25,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:25,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 12:26:25,176 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 12:26:25,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 12:26:25,177 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 12:26:25,177 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:25,177 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 12:26:25,177 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:25,177 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. started... 2023-02-13 12:26:25,177 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:26:25,177 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:25,177 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:25,177 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing adac9b9ed7bc6febc3aa79d4b3cce9dd 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:26:25,177 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:25,178 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. started... 2023-02-13 12:26:25,177 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. started... 2023-02-13 12:26:25,178 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 070d047471747182c749aaedda740bba 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:26:25,178 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4b661928fda9be3aac0c0235707673ac 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-13 12:26:25,624 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213b82b5d21b20f47f4956fc343856985c0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b82b5d21b20f47f4956fc343856985c0 2023-02-13 12:26:25,624 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213bc69a8bfea274b2d8d5dac5d16e05570 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213bc69a8bfea274b2d8d5dac5d16e05570 2023-02-13 12:26:25,627 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302130bdf83e4d9d14fccb100192fe8298667 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130bdf83e4d9d14fccb100192fe8298667 2023-02-13 12:26:25,641 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/05f3e6d083314592a8dbe487d0a6353d 2023-02-13 12:26:25,677 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213dc2acc2812104109942494320657c56e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213dc2acc2812104109942494320657c56e 2023-02-13 12:26:26,038 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/1322b7f878ed4bb2ba31dfd73a9f95d2 2023-02-13 12:26:26,038 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/6631b892db8b416d90a0998c3e26c46c 2023-02-13 12:26:26,083 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/bdb17378c1f140c6b494cc9c424d7abf 2023-02-13 12:26:26,088 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/05f3e6d083314592a8dbe487d0a6353d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d 2023-02-13 12:26:26,113 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 12:26:26,113 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/bdb17378c1f140c6b494cc9c424d7abf as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf 2023-02-13 12:26:26,117 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:26:26,117 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 070d047471747182c749aaedda740bba in 939ms, sequenceid=9, compaction requested=false 2023-02-13 12:26:26,117 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:26,117 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:26,118 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:26,482 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213902a3e2e9cbb464e8e758563caeb5f48 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213902a3e2e9cbb464e8e758563caeb5f48 2023-02-13 12:26:26,483 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f160bbb644844b2ba45fbeb39b6f6035 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213f160bbb644844b2ba45fbeb39b6f6035 2023-02-13 12:26:26,889 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/4c4eea8e9162483eaeb84752a1dba2c8 2023-02-13 12:26:26,889 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/fa576c1dc10343559b79f5915c7ffad3 2023-02-13 12:26:26,900 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/6631b892db8b416d90a0998c3e26c46c as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c 2023-02-13 12:26:26,901 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/1322b7f878ed4bb2ba31dfd73a9f95d2 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2 2023-02-13 12:26:26,926 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 12:26:26,926 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c, entries=4, sequenceid=9, filesize=5.5 K 2023-02-13 12:26:26,927 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/fa576c1dc10343559b79f5915c7ffad3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3 2023-02-13 12:26:26,927 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/4c4eea8e9162483eaeb84752a1dba2c8 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8 2023-02-13 12:26:26,930 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:26:26,930 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3, entries=2, sequenceid=9, filesize=5.1 K 2023-02-13 12:26:26,931 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for adac9b9ed7bc6febc3aa79d4b3cce9dd in 1754ms, sequenceid=9, compaction requested=false 2023-02-13 12:26:26,931 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 4b661928fda9be3aac0c0235707673ac in 1753ms, sequenceid=9, compaction requested=false 2023-02-13 12:26:26,931 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:26,931 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:26,931 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:26,931 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 12:26:26,931 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 12:26:26,966 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,966 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:26,967 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,967 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:26,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 12:26:26,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@315d7999[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:26,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 12:26:26,969 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,974 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:26,975 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 12:26:26,975 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 12:26:26,975 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:26,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,975 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:26,983 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 12:26:26,983 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,983 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:26,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:26,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:26,983 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 12:26:26,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:26,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:26,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:26,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:26,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:26,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:26,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 12:26:26,989 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 12:26:26,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:26,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:26,989 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 12:26:26,989 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:26,999 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,000 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,000 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:27,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:27,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:27,001 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:27,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:27,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:27,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:27,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:27,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:27,005 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:27,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:27,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:27,016 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:27,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 12:26:27,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:27,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:27,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:27,016 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:27,017 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:27,017 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:27,017 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:27,017 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:27,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:27,117 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:27,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:27,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:27,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:27,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:27,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:27,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:27,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 12:26:27,132 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 12:26:27,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:27,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 12:26:27,132 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 12:26:27,133 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,134 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:27,141 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:27,141 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:27,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:27,142 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:27,143 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:27,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:27,145 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 12:26:27,145 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 12:26:27,145 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 12:26:27,145 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 12:26:27,145 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:27,145 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. started... 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:27,146 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:26:27,146 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. started... 2023-02-13 12:26:27,146 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing adac9b9ed7bc6febc3aa79d4b3cce9dd 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:26:27,147 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 070d047471747182c749aaedda740bba 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:26:27,146 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. started... 2023-02-13 12:26:27,147 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4b661928fda9be3aac0c0235707673ac 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-13 12:26:27,600 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213b61f49b61bdc40a685a5a319106d781b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b61f49b61bdc40a685a5a319106d781b 2023-02-13 12:26:27,610 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302137b64ab7260f54b53a51373a529c4d90e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302137b64ab7260f54b53a51373a529c4d90e 2023-02-13 12:26:27,610 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302130387cd24e2fa4a3399d8d81670cfbbbb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130387cd24e2fa4a3399d8d81670cfbbbb 2023-02-13 12:26:27,988 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 12:26:28,012 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/54efbee33b3f4c7498b412d61552f972 2023-02-13 12:26:28,022 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/89cbec7ef5884f9ba20ff5ab36028a21 2023-02-13 12:26:28,023 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/bcf19ad436ec430f866bbb37b286b650 2023-02-13 12:26:28,460 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302136f9a1a4ab2864a458325236b72acd15c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302136f9a1a4ab2864a458325236b72acd15c 2023-02-13 12:26:28,464 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230213f589b99655ba428e8e06561859424966 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230213f589b99655ba428e8e06561859424966 2023-02-13 12:26:28,466 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213e88a59f1b8804f37ae6c420904ace8b4 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213e88a59f1b8804f37ae6c420904ace8b4 2023-02-13 12:26:28,467 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/d54063791c594f5db1dfc68d4e5e1ca3 2023-02-13 12:26:28,468 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/2ca09060831841e485f90df46d421e64 2023-02-13 12:26:28,471 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/54efbee33b3f4c7498b412d61552f972 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972 2023-02-13 12:26:28,471 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/ae45780813614e52b7c2357f03f52224 2023-02-13 12:26:28,472 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/89cbec7ef5884f9ba20ff5ab36028a21 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21 2023-02-13 12:26:28,474 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/bcf19ad436ec430f866bbb37b286b650 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650 2023-02-13 12:26:28,491 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 12:26:28,492 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 12:26:28,492 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/2ca09060831841e485f90df46d421e64 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64 2023-02-13 12:26:28,492 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/d54063791c594f5db1dfc68d4e5e1ca3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3 2023-02-13 12:26:28,493 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650, entries=4, sequenceid=13, filesize=5.5 K 2023-02-13 12:26:28,493 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/ae45780813614e52b7c2357f03f52224 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224 2023-02-13 12:26:28,497 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:26:28,497 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:26:28,497 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224, entries=2, sequenceid=13, filesize=5.1 K 2023-02-13 12:26:28,498 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 4b661928fda9be3aac0c0235707673ac in 1351ms, sequenceid=13, compaction requested=true 2023-02-13 12:26:28,498 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 070d047471747182c749aaedda740bba in 1351ms, sequenceid=13, compaction requested=true 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:28,498 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for adac9b9ed7bc6febc3aa79d4b3cce9dd in 1352ms, sequenceid=13, compaction requested=true 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:28,498 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 12:26:28,498 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 12:26:28,533 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,533 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:28,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:28,533 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,533 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:28,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:28,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 12:26:28,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66c90e3d[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:28,537 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 12:26:28,537 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,547 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,547 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 12:26:28,547 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 12:26:28,547 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:28,547 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,547 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:28,558 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 12:26:28,558 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:28,558 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 12:26:28,558 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:28,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:28,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:28,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:28,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:28,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:28,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 12:26:28,567 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 12:26:28,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:28,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:28,567 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 12:26:28,567 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:28,574 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,574 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:28,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:28,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,575 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:28,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:28,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:28,576 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:28,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,578 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:28,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:28,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,604 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:28,604 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:28,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:28,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 12:26:28,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:28,604 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,604 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:28,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 12:26:28,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:28,604 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:28,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:28,606 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:28,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:28,705 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:28,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:28,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:28,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:28,711 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. with WAL disabled. Data may be lost in the event of a crash. 2023-02-13 12:26:28,718 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:28,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:28,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 12:26:28,721 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 12:26:28,721 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:28,721 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 12:26:28,721 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 12:26:28,722 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,722 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:28,733 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:28,733 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:28,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:28,734 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:28,734 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:28,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:28,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:28,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:28,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 12:26:28,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 12:26:28,738 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 12:26:28,738 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 12:26:28,738 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:28,738 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 12:26:28,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:28,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. started... 2023-02-13 12:26:28,739 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:26:28,739 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:28,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:28,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. started... 2023-02-13 12:26:28,739 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing adac9b9ed7bc6febc3aa79d4b3cce9dd 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:26:28,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:28,739 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4b661928fda9be3aac0c0235707673ac 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:26:28,740 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. started... 2023-02-13 12:26:28,740 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 070d047471747182c749aaedda740bba 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-13 12:26:28,769 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:26:28,769 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x464c4440 to 127.0.0.1:64508 2023-02-13 12:26:28,769 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:28,769 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:26:28,770 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29ece623 to 127.0.0.1:64508 2023-02-13 12:26:28,770 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:29,213 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213c0af3d66a4c945689d9c65bc61a717a8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213c0af3d66a4c945689d9c65bc61a717a8 2023-02-13 12:26:29,213 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302131c086ce116474944b11eff465b25a822 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302131c086ce116474944b11eff465b25a822 2023-02-13 12:26:29,220 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302136b4c53827d1a450b82595bb24fcfc7c9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302136b4c53827d1a450b82595bb24fcfc7c9 2023-02-13 12:26:29,229 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/c3653f2f58b748af92489a206c0b6c16 2023-02-13 12:26:29,629 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/40c3db96de704952bab719cabfde9912 2023-02-13 12:26:29,635 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/621e0fab5139496b9a065e946c4dddc3 2023-02-13 12:26:29,660 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302130874b65db7b541d6b79c68ff12341643 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302130874b65db7b541d6b79c68ff12341643 2023-02-13 12:26:29,664 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/33d6d31ead8c4c1499376dc2d8ac14f9 2023-02-13 12:26:29,668 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/c3653f2f58b748af92489a206c0b6c16 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16 2023-02-13 12:26:29,671 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213cf5a98d9029b4ecbaaa135d0d0ad7e60 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213cf5a98d9029b4ecbaaa135d0d0ad7e60 2023-02-13 12:26:29,691 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 12:26:29,691 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/33d6d31ead8c4c1499376dc2d8ac14f9 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9 2023-02-13 12:26:29,695 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:26:29,696 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 070d047471747182c749aaedda740bba in 956ms, sequenceid=17, compaction requested=true 2023-02-13 12:26:29,696 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:29,696 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:29,696 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:30,078 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/4897bc66c4b94380a911af1f13cb34c5 2023-02-13 12:26:30,089 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/40c3db96de704952bab719cabfde9912 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912 2023-02-13 12:26:30,091 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302137efa542b029e481abcf8863f95c42c16 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302137efa542b029e481abcf8863f95c42c16 2023-02-13 12:26:30,116 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 12:26:30,116 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/4897bc66c4b94380a911af1f13cb34c5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5 2023-02-13 12:26:30,120 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:26:30,120 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 4b661928fda9be3aac0c0235707673ac in 1381ms, sequenceid=17, compaction requested=true 2023-02-13 12:26:30,121 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:30,121 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:30,121 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:30,498 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/67ad01494d5e4c5bbb4eb3feb99773b0 2023-02-13 12:26:30,510 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/621e0fab5139496b9a065e946c4dddc3 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3 2023-02-13 12:26:30,534 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3, entries=4, sequenceid=17, filesize=5.5 K 2023-02-13 12:26:30,535 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/67ad01494d5e4c5bbb4eb3feb99773b0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0 2023-02-13 12:26:30,539 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0, entries=2, sequenceid=17, filesize=5.1 K 2023-02-13 12:26:30,540 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for adac9b9ed7bc6febc3aa79d4b3cce9dd in 1801ms, sequenceid=17, compaction requested=true 2023-02-13 12:26:30,540 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:30,540 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:30,540 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:30,540 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:30,540 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,540 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 12:26:30,540 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 12:26:30,574 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,574 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:30,575 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:30,575 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:30,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:30,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:30,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 12:26:30,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10f83[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:30,577 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 12:26:30,577 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,583 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,583 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 12:26:30,583 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 12:26:30,583 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:30,583 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,583 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:30,591 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 12:26:30,591 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:30,591 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 12:26:30,591 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:30,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:30,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:30,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:30,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:30,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:30,595 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 12:26:30,595 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 12:26:30,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:30,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:30,596 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 12:26:30,596 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:30,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:30,605 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:30,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:30,606 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:30,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,609 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:30,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 12:26:30,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:30,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:30,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:30,616 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:30,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,617 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:30,617 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:30,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:30,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:30,617 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,618 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:30,618 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,718 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:30,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:30,735 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:30,743 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:30,751 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:30,751 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:30,831 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 12:26:30,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-13 12:26:30,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-13 12:26:30,836 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-13 12:26:30,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:30,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-13 12:26:30,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-13 12:26:30,836 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,837 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:30,846 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:30,846 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:30,847 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:30,847 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:30,847 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-13 12:26:30,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-13 12:26:30,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-13 12:26:30,848 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-13 12:26:30,848 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 12:26:30,848 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:30,848 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-13 12:26:30,848 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. started... 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. started... 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:30,848 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. started... 2023-02-13 12:26:30,848 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing adac9b9ed7bc6febc3aa79d4b3cce9dd 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:26:30,848 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 070d047471747182c749aaedda740bba 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:26:30,848 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 4b661928fda9be3aac0c0235707673ac 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-13 12:26:30,884 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/42aa3b7b2daa4e45b87a0530f13b2fa1 2023-02-13 12:26:30,904 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42aa3b7b2daa4e45b87a0530f13b2fa1 2023-02-13 12:26:31,286 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/25daaab740eb4304b8877b94cc0668ed 2023-02-13 12:26:31,305 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25daaab740eb4304b8877b94cc0668ed 2023-02-13 12:26:31,693 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/5f457d4941934122b4ec8348460d8e1f 2023-02-13 12:26:31,714 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/9e32d1ddd9fc4af68d43ccac835c2739 2023-02-13 12:26:31,717 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e32d1ddd9fc4af68d43ccac835c2739 2023-02-13 12:26:31,718 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/42aa3b7b2daa4e45b87a0530f13b2fa1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1 2023-02-13 12:26:31,724 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f457d4941934122b4ec8348460d8e1f 2023-02-13 12:26:31,739 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 42aa3b7b2daa4e45b87a0530f13b2fa1 2023-02-13 12:26:31,739 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 12:26:31,739 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/9e32d1ddd9fc4af68d43ccac835c2739 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739 2023-02-13 12:26:31,742 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9e32d1ddd9fc4af68d43ccac835c2739 2023-02-13 12:26:31,742 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:31,743 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 4b661928fda9be3aac0c0235707673ac in 895ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:31,743 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:31,743 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:31,743 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-13 12:26:32,116 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/e9cbcaa1e5894e989ac3cd5f7624797e 2023-02-13 12:26:32,127 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9cbcaa1e5894e989ac3cd5f7624797e 2023-02-13 12:26:32,128 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/25daaab740eb4304b8877b94cc0668ed as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed 2023-02-13 12:26:32,155 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 25daaab740eb4304b8877b94cc0668ed 2023-02-13 12:26:32,155 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 12:26:32,156 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/e9cbcaa1e5894e989ac3cd5f7624797e as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e 2023-02-13 12:26:32,159 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9cbcaa1e5894e989ac3cd5f7624797e 2023-02-13 12:26:32,159 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:32,160 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 070d047471747182c749aaedda740bba in 1312ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:32,160 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:32,160 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:32,160 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-13 12:26:32,538 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/a9f9dbce3dec446f8d0d5dac6c8909e5 2023-02-13 12:26:32,549 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9f9dbce3dec446f8d0d5dac6c8909e5 2023-02-13 12:26:32,550 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/5f457d4941934122b4ec8348460d8e1f as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f 2023-02-13 12:26:32,573 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5f457d4941934122b4ec8348460d8e1f 2023-02-13 12:26:32,573 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f, entries=3, sequenceid=23, filesize=5.2 K 2023-02-13 12:26:32,574 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/a9f9dbce3dec446f8d0d5dac6c8909e5 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5 2023-02-13 12:26:32,577 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9f9dbce3dec446f8d0d5dac6c8909e5 2023-02-13 12:26:32,577 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5, entries=1, sequenceid=23, filesize=5.0 K 2023-02-13 12:26:32,578 INFO [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for adac9b9ed7bc6febc3aa79d4b3cce9dd in 1730ms, sequenceid=23, compaction requested=true 2023-02-13 12:26:32,578 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:32,578 DEBUG [rs(jenkins-hbase16.apache.org,38471,1676291011293)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:32,578 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-13 12:26:32,578 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-13 12:26:32,578 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,578 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-13 12:26:32,578 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-13 12:26:32,607 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,607 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:32,608 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,608 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 12:26:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:32,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,38471,1676291011293' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-13 12:26:32,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17e0a512[Count = 0] remaining members to acquire global barrier 2023-02-13 12:26:32,611 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-13 12:26:32,611 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,616 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,616 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-13 12:26:32,616 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-13 12:26:32,616 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase16.apache.org,38471,1676291011293' in zk 2023-02-13 12:26:32,616 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,616 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 12:26:32,624 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-13 12:26:32,624 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,624 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:32,624 DEBUG [member: 'jenkins-hbase16.apache.org,38471,1676291011293' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-13 12:26:32,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:32,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:32,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase16.apache.org,38471,1676291011293': 2023-02-13 12:26:32,631 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,38471,1676291011293' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-13 12:26:32,631 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-13 12:26:32,631 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 12:26:32,631 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 12:26:32,631 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-13 12:26:32,631 INFO [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 12:26:32,641 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,641 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 12:26:32,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 12:26:32,641 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:32,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:32,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:32,642 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 12:26:32,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 12:26:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,644 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 12:26:32,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-13 12:26:32,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-13 12:26:32,655 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-13 12:26:32,655 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:32,655 DEBUG [(jenkins-hbase16.apache.org,41493,1676291010224)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 12:26:32,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 12:26:32,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 12:26:32,655 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-13 12:26:32,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-13 12:26:32,656 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-13 12:26:32,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 12:26:32,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 12:26:32,755 DEBUG [Listener at localhost.localdomain/33629] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 12:26:32,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41493] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 12:26:32,758 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:32,758 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 is initiating major compaction (all files) 2023-02-13 12:26:32,758 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:32,758 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:32,759 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 in testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:32,759 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp, totalSize=27.1 K 2023-02-13 12:26:32,760 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 56e050ca9de34f609611048c278b3ed0, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183150 2023-02-13 12:26:32,761 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 1322b7f878ed4bb2ba31dfd73a9f95d2, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185159 2023-02-13 12:26:32,762 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 89cbec7ef5884f9ba20ff5ab36028a21, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187121 2023-02-13 12:26:32,763 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 621e0fab5139496b9a065e946c4dddc3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188710 2023-02-13 12:26:32,764 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 5f457d4941934122b4ec8348460d8e1f, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:32,785 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): adac9b9ed7bc6febc3aa79d4b3cce9dd#family1#compaction#198 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:33,229 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del 2023-02-13 12:26:33,229 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del 2023-02-13 12:26:34,091 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e428a7c4ba8249d6b8c1b47f174edd4b 2023-02-13 12:26:34,091 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family1/e428a7c4ba8249d6b8c1b47f174edd4b as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/e428a7c4ba8249d6b8c1b47f174edd4b 2023-02-13 12:26:34,110 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e428a7c4ba8249d6b8c1b47f174edd4b 2023-02-13 12:26:34,111 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 of adac9b9ed7bc6febc3aa79d4b3cce9dd into e428a7c4ba8249d6b8c1b47f174edd4b(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:34,111 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:34,111 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:34,111 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): adac9b9ed7bc6febc3aa79d4b3cce9dd/family2 is initiating major compaction (all files) 2023-02-13 12:26:34,111 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:34,111 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:34,111 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of adac9b9ed7bc6febc3aa79d4b3cce9dd/family2 in testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:34,111 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp, totalSize=25.4 K 2023-02-13 12:26:34,112 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 8314e500fcb74be7a31916859f48fca2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183150 2023-02-13 12:26:34,112 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting fa576c1dc10343559b79f5915c7ffad3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185159 2023-02-13 12:26:34,113 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 2ca09060831841e485f90df46d421e64, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187121 2023-02-13 12:26:34,113 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 67ad01494d5e4c5bbb4eb3feb99773b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188710 2023-02-13 12:26:34,113 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting a9f9dbce3dec446f8d0d5dac6c8909e5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:34,119 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): adac9b9ed7bc6febc3aa79d4b3cce9dd#family2#compaction#199 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:34,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f] to archive 2023-02-13 12:26:34,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:34,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/56e050ca9de34f609611048c278b3ed0 2023-02-13 12:26:34,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/1322b7f878ed4bb2ba31dfd73a9f95d2 2023-02-13 12:26:34,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/89cbec7ef5884f9ba20ff5ab36028a21 2023-02-13 12:26:34,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/621e0fab5139496b9a065e946c4dddc3 2023-02-13 12:26:34,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/5f457d4941934122b4ec8348460d8e1f 2023-02-13 12:26:34,536 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302138b04687f7f4043f3b9828690c564c8a1_del 2023-02-13 12:26:34,536 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302138b04687f7f4043f3b9828690c564c8a1_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302138b04687f7f4043f3b9828690c564c8a1_del 2023-02-13 12:26:35,367 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d92f9507b7074234919d5fc560059d00 2023-02-13 12:26:35,368 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/.tmp/family2/d92f9507b7074234919d5fc560059d00 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/d92f9507b7074234919d5fc560059d00 2023-02-13 12:26:35,372 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d92f9507b7074234919d5fc560059d00 2023-02-13 12:26:35,373 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in adac9b9ed7bc6febc3aa79d4b3cce9dd/family2 of adac9b9ed7bc6febc3aa79d4b3cce9dd into d92f9507b7074234919d5fc560059d00(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:35,374 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:35,374 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:35,374 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 4b661928fda9be3aac0c0235707673ac/family1 is initiating major compaction (all files) 2023-02-13 12:26:35,374 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:35,374 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:35,375 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 4b661928fda9be3aac0c0235707673ac/family1 in testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:35,375 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp, totalSize=27.1 K 2023-02-13 12:26:35,375 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f36fa54339184154b1d39c2a0b5b13b4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183149 2023-02-13 12:26:35,376 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 6631b892db8b416d90a0998c3e26c46c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185158 2023-02-13 12:26:35,376 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 54efbee33b3f4c7498b412d61552f972, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187121 2023-02-13 12:26:35,377 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 40c3db96de704952bab719cabfde9912, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188709 2023-02-13 12:26:35,377 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 42aa3b7b2daa4e45b87a0530f13b2fa1, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:35,385 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 4b661928fda9be3aac0c0235707673ac#family1#compaction#200 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:35,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5] to archive 2023-02-13 12:26:35,424 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:35,426 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/8314e500fcb74be7a31916859f48fca2 2023-02-13 12:26:35,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/fa576c1dc10343559b79f5915c7ffad3 2023-02-13 12:26:35,429 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/2ca09060831841e485f90df46d421e64 2023-02-13 12:26:35,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/67ad01494d5e4c5bbb4eb3feb99773b0 2023-02-13 12:26:35,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family2/a9f9dbce3dec446f8d0d5dac6c8909e5 2023-02-13 12:26:35,827 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del 2023-02-13 12:26:35,827 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del 2023-02-13 12:26:36,692 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c8b5f596773413dbc4756171d8275bf 2023-02-13 12:26:36,693 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family1/0c8b5f596773413dbc4756171d8275bf as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/0c8b5f596773413dbc4756171d8275bf 2023-02-13 12:26:36,710 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c8b5f596773413dbc4756171d8275bf 2023-02-13 12:26:36,711 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4b661928fda9be3aac0c0235707673ac/family1 of 4b661928fda9be3aac0c0235707673ac into 0c8b5f596773413dbc4756171d8275bf(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:36,711 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:36,711 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:36,711 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 4b661928fda9be3aac0c0235707673ac/family2 is initiating major compaction (all files) 2023-02-13 12:26:36,711 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:36,711 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:36,711 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 4b661928fda9be3aac0c0235707673ac/family2 in testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:36,711 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp, totalSize=25.4 K 2023-02-13 12:26:36,712 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 72f2d27cf2034d63b637e41ec3478325, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183149 2023-02-13 12:26:36,712 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 4c4eea8e9162483eaeb84752a1dba2c8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185158 2023-02-13 12:26:36,712 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting d54063791c594f5db1dfc68d4e5e1ca3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187121 2023-02-13 12:26:36,713 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 4897bc66c4b94380a911af1f13cb34c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188709 2023-02-13 12:26:36,713 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 9e32d1ddd9fc4af68d43ccac835c2739, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:36,718 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 4b661928fda9be3aac0c0235707673ac#family2#compaction#201 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:36,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1] to archive 2023-02-13 12:26:36,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:36,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/f36fa54339184154b1d39c2a0b5b13b4 2023-02-13 12:26:36,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/6631b892db8b416d90a0998c3e26c46c 2023-02-13 12:26:36,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/54efbee33b3f4c7498b412d61552f972 2023-02-13 12:26:36,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/40c3db96de704952bab719cabfde9912 2023-02-13 12:26:36,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/42aa3b7b2daa4e45b87a0530f13b2fa1 2023-02-13 12:26:37,133 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230213e5ee4ed14d644e368a692f7ad510c2c8_del 2023-02-13 12:26:37,133 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230213e5ee4ed14d644e368a692f7ad510c2c8_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230213e5ee4ed14d644e368a692f7ad510c2c8_del 2023-02-13 12:26:37,554 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4711ebb1c98f4269a31eb364d65c8ccb 2023-02-13 12:26:37,555 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/.tmp/family2/4711ebb1c98f4269a31eb364d65c8ccb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4711ebb1c98f4269a31eb364d65c8ccb 2023-02-13 12:26:37,561 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4711ebb1c98f4269a31eb364d65c8ccb 2023-02-13 12:26:37,562 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 4b661928fda9be3aac0c0235707673ac/family2 of 4b661928fda9be3aac0c0235707673ac into 4711ebb1c98f4269a31eb364d65c8ccb(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:26:37,562 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:37,562 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:37,562 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 070d047471747182c749aaedda740bba/family1 is initiating major compaction (all files) 2023-02-13 12:26:37,563 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:37,563 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:37,563 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 070d047471747182c749aaedda740bba/family1 in testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:37,563 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp, totalSize=27.1 K 2023-02-13 12:26:37,564 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting f0b1ca6f96a14adf85ea38d8cc8ef7c5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183151 2023-02-13 12:26:37,564 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 05f3e6d083314592a8dbe487d0a6353d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185159 2023-02-13 12:26:37,565 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting bcf19ad436ec430f866bbb37b286b650, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187122 2023-02-13 12:26:37,566 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting c3653f2f58b748af92489a206c0b6c16, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188710 2023-02-13 12:26:37,566 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 25daaab740eb4304b8877b94cc0668ed, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:37,580 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 070d047471747182c749aaedda740bba#family1#compaction#202 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:37,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739] to archive 2023-02-13 12:26:37,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:37,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/72f2d27cf2034d63b637e41ec3478325 2023-02-13 12:26:37,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4c4eea8e9162483eaeb84752a1dba2c8 2023-02-13 12:26:37,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/d54063791c594f5db1dfc68d4e5e1ca3 2023-02-13 12:26:37,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/4897bc66c4b94380a911af1f13cb34c5 2023-02-13 12:26:37,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family2/9e32d1ddd9fc4af68d43ccac835c2739 2023-02-13 12:26:38,026 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del 2023-02-13 12:26:38,026 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del 2023-02-13 12:26:38,909 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccde9d3c49f54748a0a74826157579cb 2023-02-13 12:26:38,910 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family1/ccde9d3c49f54748a0a74826157579cb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/ccde9d3c49f54748a0a74826157579cb 2023-02-13 12:26:38,927 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ccde9d3c49f54748a0a74826157579cb 2023-02-13 12:26:38,928 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 070d047471747182c749aaedda740bba/family1 of 070d047471747182c749aaedda740bba into ccde9d3c49f54748a0a74826157579cb(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-13 12:26:38,928 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:38,928 DEBUG [Listener at localhost.localdomain/33629] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-13 12:26:38,928 DEBUG [Listener at localhost.localdomain/33629] regionserver.HStore(1912): 070d047471747182c749aaedda740bba/family2 is initiating major compaction (all files) 2023-02-13 12:26:38,928 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 12:26:38,928 INFO [Listener at localhost.localdomain/33629] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 12:26:38,929 INFO [Listener at localhost.localdomain/33629] regionserver.HRegion(2258): Starting compaction of 070d047471747182c749aaedda740bba/family2 in testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:38,929 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e] into tmpdir=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp, totalSize=25.4 K 2023-02-13 12:26:38,930 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting fe1e7cd7ed7d4dc48f39aadc92eaf967, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676291183151 2023-02-13 12:26:38,930 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting bdb17378c1f140c6b494cc9c424d7abf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676291185159 2023-02-13 12:26:38,930 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting ae45780813614e52b7c2357f03f52224, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676291187122 2023-02-13 12:26:38,931 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting 33d6d31ead8c4c1499376dc2d8ac14f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676291188710 2023-02-13 12:26:38,931 DEBUG [Listener at localhost.localdomain/33629] compactions.Compactor(207): Compacting e9cbcaa1e5894e989ac3cd5f7624797e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-13 12:26:38,938 INFO [Listener at localhost.localdomain/33629] throttle.PressureAwareThroughputController(145): 070d047471747182c749aaedda740bba#family2#compaction#203 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 12:26:39,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed] to archive 2023-02-13 12:26:39,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:39,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/f0b1ca6f96a14adf85ea38d8cc8ef7c5 2023-02-13 12:26:39,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/05f3e6d083314592a8dbe487d0a6353d 2023-02-13 12:26:39,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/bcf19ad436ec430f866bbb37b286b650 2023-02-13 12:26:39,033 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/c3653f2f58b748af92489a206c0b6c16 2023-02-13 12:26:39,034 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/25daaab740eb4304b8877b94cc0668ed 2023-02-13 12:26:39,352 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230213be0b31b7a74c4a8e82c6da1ffe4c1f8b_del 2023-02-13 12:26:39,352 INFO [Listener at localhost.localdomain/33629] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213be0b31b7a74c4a8e82c6da1ffe4c1f8b_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230213be0b31b7a74c4a8e82c6da1ffe4c1f8b_del 2023-02-13 12:26:39,776 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb2dd1c3e02c4ee5a97f39ab93339e40 2023-02-13 12:26:39,776 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/.tmp/family2/fb2dd1c3e02c4ee5a97f39ab93339e40 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fb2dd1c3e02c4ee5a97f39ab93339e40 2023-02-13 12:26:39,779 INFO [Listener at localhost.localdomain/33629] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fb2dd1c3e02c4ee5a97f39ab93339e40 2023-02-13 12:26:39,780 INFO [Listener at localhost.localdomain/33629] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 070d047471747182c749aaedda740bba/family2 of 070d047471747182c749aaedda740bba into fb2dd1c3e02c4ee5a97f39ab93339e40(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 12:26:39,780 DEBUG [Listener at localhost.localdomain/33629] regionserver.HRegion(2288): Compaction status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:39,784 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:39,787 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:39,790 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:39,791 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:39,797 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41493] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x08\x8B\xA5\xF2\x1CV\x07\x1CN\xA4:\xB9\x8Bq&4"\x10f5\xD7&\x0B\x93\x9F{\x97\xF6ps\x09\x15\xDDK*\x10!nl\x01 \x1C\x88\xD0m\x98EF&\x0C\xA5t2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-13 12:26:39,816 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 12:26:39,816 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 12:26:39,817 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:26:39,817 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676291799817 2023-02-13 12:26:39,818 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-13 12:26:39,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e] to archive 2023-02-13 12:26:39,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:39,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/fe1e7cd7ed7d4dc48f39aadc92eaf967 2023-02-13 12:26:39,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/bdb17378c1f140c6b494cc9c424d7abf 2023-02-13 12:26:39,822 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/ae45780813614e52b7c2357f03f52224 2023-02-13 12:26:39,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/33d6d31ead8c4c1499376dc2d8ac14f9 2023-02-13 12:26:39,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family2/e9cbcaa1e5894e989ac3cd5f7624797e 2023-02-13 12:26:40,080 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-13 12:26:40,099 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del 2023-02-13 12:26:40,118 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del 2023-02-13 12:26:40,137 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del 2023-02-13 12:26:40,137 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:40,137 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] zookeeper.ReadOnlyZKClient(139): Connect 0x250c3d7e to 127.0.0.1:64508 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 12:26:40,192 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f50cd5f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 12:26:40,192 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:40,681 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 12:26:41,140 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021381b8142915c34515a7e26016d64bccf6 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021381b8142915c34515a7e26016d64bccf6 2023-02-13 12:26:41,141 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:41,155 DEBUG [MobCompactor-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 12:26:41,157 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:50468, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 12:26:41,166 DEBUG [MobCompactor-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 12:26:41,167 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:37932, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 12:26:41,191 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/1f53f057737347e9a91906a1682906f0 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-13 12:26:41,192 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/1f53f057737347e9a91906a1682906f0}] 2023-02-13 12:26:41,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/1f53f057737347e9a91906a1682906f0 for inclusion in adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 2023-02-13 12:26:41,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-13 12:26:41,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-13 12:26:41,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: 2023-02-13 12:26:41,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213/family1/1f53f057737347e9a91906a1682906f0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__9qr2f5mmojl61573njn17i2hu3c1nnac67bm4q0d9iilljdafnkbccoek80tk450/family1/1f53f057737347e9a91906a1682906f0 2023-02-13 12:26:41,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__9qr2f5mmojl61573njn17i2hu3c1nnac67bm4q0d9iilljdafnkbccoek80tk450/family1/1f53f057737347e9a91906a1682906f0 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/c7980ce7f1394a9da69e671dcc8296bc_SeqId_28_ 2023-02-13 12:26:41,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__9qr2f5mmojl61573njn17i2hu3c1nnac67bm4q0d9iilljdafnkbccoek80tk450/family1/1f53f057737347e9a91906a1682906f0 into adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/c7980ce7f1394a9da69e671dcc8296bc_SeqId_28_ - updating store file list. 2023-02-13 12:26:41,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/c7980ce7f1394a9da69e671dcc8296bc_SeqId_28_ into adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 2023-02-13 12:26:41,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__9qr2f5mmojl61573njn17i2hu3c1nnac67bm4q0d9iilljdafnkbccoek80tk450/family1/1f53f057737347e9a91906a1682906f0 into adac9b9ed7bc6febc3aa79d4b3cce9dd/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/family1/c7980ce7f1394a9da69e671dcc8296bc_SeqId_28_) 2023-02-13 12:26:41,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__9qr2f5mmojl61573njn17i2hu3c1nnac67bm4q0d9iilljdafnkbccoek80tk450/family1/1f53f057737347e9a91906a1682906f0 2023-02-13 12:26:41,241 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:41,242 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:41,243 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130387cd24e2fa4a3399d8d81670cfbbbb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302130387cd24e2fa4a3399d8d81670cfbbbb 2023-02-13 12:26:41,244 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302136b4c53827d1a450b82595bb24fcfc7c9 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302136b4c53827d1a450b82595bb24fcfc7c9 2023-02-13 12:26:41,245 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213bc69a8bfea274b2d8d5dac5d16e05570 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213bc69a8bfea274b2d8d5dac5d16e05570 2023-02-13 12:26:41,246 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213de1acb571ece4c3ea19db6ecf58dc326 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230213de1acb571ece4c3ea19db6ecf58dc326 2023-02-13 12:26:41,246 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 12:26:41,246 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230213' 2023-02-13 12:26:41,246 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:41,246 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:41,547 DEBUG [regionserver/jenkins-hbase16:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 12:26:42,177 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230213290f2a0ded97458494227984d9ac77bc to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230213290f2a0ded97458494227984d9ac77bc 2023-02-13 12:26:42,178 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:42,222 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/f4fd75deb9ed4b1991effd54b11f25ad first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-13 12:26:42,223 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/f4fd75deb9ed4b1991effd54b11f25ad}] 2023-02-13 12:26:42,225 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/f4fd75deb9ed4b1991effd54b11f25ad for inclusion in 070d047471747182c749aaedda740bba/family1 2023-02-13 12:26:42,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-13 12:26:42,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-13 12:26:42,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 070d047471747182c749aaedda740bba: 2023-02-13 12:26:42,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213/family1/f4fd75deb9ed4b1991effd54b11f25ad to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__dia7h169of1o3qeril7i94esbsljesajqmjnbe493qf4a8r5roeg69c78sg5o8ji/family1/f4fd75deb9ed4b1991effd54b11f25ad 2023-02-13 12:26:42,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__dia7h169of1o3qeril7i94esbsljesajqmjnbe493qf4a8r5roeg69c78sg5o8ji/family1/f4fd75deb9ed4b1991effd54b11f25ad as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/5db1e0736fa5420aa7da0040e92486e7_SeqId_28_ 2023-02-13 12:26:42,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__dia7h169of1o3qeril7i94esbsljesajqmjnbe493qf4a8r5roeg69c78sg5o8ji/family1/f4fd75deb9ed4b1991effd54b11f25ad into 070d047471747182c749aaedda740bba/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/5db1e0736fa5420aa7da0040e92486e7_SeqId_28_ - updating store file list. 2023-02-13 12:26:42,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/5db1e0736fa5420aa7da0040e92486e7_SeqId_28_ into 070d047471747182c749aaedda740bba/family1 2023-02-13 12:26:42,266 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__dia7h169of1o3qeril7i94esbsljesajqmjnbe493qf4a8r5roeg69c78sg5o8ji/family1/f4fd75deb9ed4b1991effd54b11f25ad into 070d047471747182c749aaedda740bba/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/family1/5db1e0736fa5420aa7da0040e92486e7_SeqId_28_) 2023-02-13 12:26:42,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__dia7h169of1o3qeril7i94esbsljesajqmjnbe493qf4a8r5roeg69c78sg5o8ji/family1/f4fd75deb9ed4b1991effd54b11f25ad 2023-02-13 12:26:42,268 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:42,269 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:42,270 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130bdf83e4d9d14fccb100192fe8298667 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302130bdf83e4d9d14fccb100192fe8298667 2023-02-13 12:26:42,271 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302131c086ce116474944b11eff465b25a822 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302131c086ce116474944b11eff465b25a822 2023-02-13 12:26:42,272 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302133abb0d81b4d04aaa8ed5a1420bd75262 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302133abb0d81b4d04aaa8ed5a1420bd75262 2023-02-13 12:26:42,273 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302137b64ab7260f54b53a51373a529c4d90e to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302137b64ab7260f54b53a51373a529c4d90e 2023-02-13 12:26:42,273 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 12:26:42,273 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230213' 2023-02-13 12:26:42,273 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:42,273 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:43,215 INFO [MobCompactor-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302133e46b6544297415c8e7661a159a87eab to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302133e46b6544297415c8e7661a159a87eab 2023-02-13 12:26:43,216 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:43,252 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65882379368e4f83b7d842fd94517e91 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-13 12:26:43,253 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac., hostname=jenkins-hbase16.apache.org,38471,1676291011293, seqNum=2 for row with hfile group [{family1,hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65882379368e4f83b7d842fd94517e91}] 2023-02-13 12:26:43,254 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(763): Validating hfile at hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65882379368e4f83b7d842fd94517e91 for inclusion in 4b661928fda9be3aac0c0235707673ac/family1 2023-02-13 12:26:43,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-13 12:26:43,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(777): Region bounds: first= last=1 2023-02-13 12:26:43,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegion(2445): Flush status journal for 4b661928fda9be3aac0c0235707673ac: 2023-02-13 12:26:43,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213/family1/65882379368e4f83b7d842fd94517e91 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__ei95pos5rc5il9s3jb5df80gdtnefnq3eidogpvp86m1eukf9njs5tg6c0c0nmsc/family1/65882379368e4f83b7d842fd94517e91 2023-02-13 12:26:43,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__ei95pos5rc5il9s3jb5df80gdtnefnq3eidogpvp86m1eukf9njs5tg6c0c0nmsc/family1/65882379368e4f83b7d842fd94517e91 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/80ecb28ff1e6408fa508c8c8c9887ca9_SeqId_28_ 2023-02-13 12:26:43,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(849): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__ei95pos5rc5il9s3jb5df80gdtnefnq3eidogpvp86m1eukf9njs5tg6c0c0nmsc/family1/65882379368e4f83b7d842fd94517e91 into 4b661928fda9be3aac0c0235707673ac/family1 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/80ecb28ff1e6408fa508c8c8c9887ca9_SeqId_28_ - updating store file list. 2023-02-13 12:26:43,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(882): Loaded HFile hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/80ecb28ff1e6408fa508c8c8c9887ca9_SeqId_28_ into 4b661928fda9be3aac0c0235707673ac/family1 2023-02-13 12:26:43,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.HStore(855): Successfully loaded hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__ei95pos5rc5il9s3jb5df80gdtnefnq3eidogpvp86m1eukf9njs5tg6c0c0nmsc/family1/65882379368e4f83b7d842fd94517e91 into 4b661928fda9be3aac0c0235707673ac/family1 (new location: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/family1/80ecb28ff1e6408fa508c8c8c9887ca9_SeqId_28_) 2023-02-13 12:26:43,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38471] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/staging/jenkins__testMajorCompactionFromAdmin__ei95pos5rc5il9s3jb5df80gdtnefnq3eidogpvp86m1eukf9njs5tg6c0c0nmsc/family1/65882379368e4f83b7d842fd94517e91 2023-02-13 12:26:43,297 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230213 table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:43,298 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:43,299 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b61f49b61bdc40a685a5a319106d781b to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b61f49b61bdc40a685a5a319106d781b 2023-02-13 12:26:43,300 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b82b5d21b20f47f4956fc343856985c0 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213b82b5d21b20f47f4956fc343856985c0 2023-02-13 12:26:43,301 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213bcb77e0dcafa42c58a0b35abaa7fc1eb to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213bcb77e0dcafa42c58a0b35abaa7fc1eb 2023-02-13 12:26:43,302 DEBUG [MobCompactor-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213c0af3d66a4c945689d9c65bc61a717a8 to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230213c0af3d66a4c945689d9c65bc61a717a8 2023-02-13 12:26:43,302 INFO [MobCompactor-pool-1] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-13 12:26:43,302 DEBUG [MobCompactor-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230213' 2023-02-13 12:26:43,302 INFO [MobCompactor-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230213 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:43,302 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-13 12:26:43,303 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-13 12:26:43,303 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del] 2023-02-13 12:26:43,303 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:43,304 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302134cc621f8691c4f5b9019b122d2f6c249_del 2023-02-13 12:26:43,304 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 12:26:43,304 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del] 2023-02-13 12:26:43,305 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:43,305 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302138b3be4022e684e4db9bdbca70d9ebf7f_del 2023-02-13 12:26:43,306 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 12:26:43,306 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] compactions.PartitionedMobCompactor(367): [hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del] 2023-02-13 12:26:43,306 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 12:26:43,307 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del to hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302135b0cfaec9ce4417caff96ba6dd5ef9af_del 2023-02-13 12:26:43,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-13 12:26:43,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster-MasterMobCompaction-1676291016003] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-13 12:26:43,308 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-13 12:26:43,309 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.4930 sec 2023-02-13 12:26:43,329 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:43,332 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:43,335 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-13 12:26:43,337 DEBUG [Listener at localhost.localdomain/33629] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-13 12:26:43,342 INFO [Listener at localhost.localdomain/33629] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 12:26:43,376 INFO [Listener at localhost.localdomain/33629] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=246 (was 253), OpenFileDescriptor=624 (was 644), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=62 (was 44) - SystemLoadAverage LEAK? -, ProcessCount=166 (was 166), AvailableMemoryMB=5845 (was 5866) 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x649a9174 to 127.0.0.1:64508 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:43,377 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 12:26:43,377 INFO [Listener at localhost.localdomain/33629] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6f2ea739 to 127.0.0.1:64508 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 12:26:43,377 DEBUG [Listener at localhost.localdomain/33629] util.JVMClusterUtil(257): Found active master hash=789300458, stopped=false 2023-02-13 12:26:43,377 INFO [Listener at localhost.localdomain/33629] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:26:43,441 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:26:43,441 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 12:26:43,441 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:26:43,441 INFO [Listener at localhost.localdomain/33629] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 12:26:43,441 DEBUG [Listener at localhost.localdomain/33629] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x241e3d53 to 127.0.0.1:64508 2023-02-13 12:26:43,442 DEBUG [Listener at localhost.localdomain/33629] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:43,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:26:43,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 12:26:43,442 INFO [Listener at localhost.localdomain/33629] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,38471,1676291011293' ***** 2023-02-13 12:26:43,443 INFO [Listener at localhost.localdomain/33629] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 12:26:43,443 INFO [RS:0;jenkins-hbase16:38471] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 12:26:43,444 INFO [RS:0;jenkins-hbase16:38471] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 12:26:43,444 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 12:26:43,444 INFO [RS:0;jenkins-hbase16:38471] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 12:26:43,444 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 31a6bfc09681f3a11506a4775f610e4d 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for adac9b9ed7bc6febc3aa79d4b3cce9dd 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 4993132286c530710e08273860fde1c5 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 534c6c8744730b7c32dbd5f29d42ddff 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for be0d091bd879acd2c49cb81bc7c86ecc 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 0b6d53fe273287f42b700182304d00f6 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for e7ec1481802597d098b27d5a31dd86f7 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 73394efe2c0767e17a09a439425afd89 2023-02-13 12:26:43,445 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 4b661928fda9be3aac0c0235707673ac 2023-02-13 12:26:43,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 31a6bfc09681f3a11506a4775f610e4d, disabling compactions & flushes 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for c410f61e2e0cf5f28a4b5c4a521e8eb0 2023-02-13 12:26:43,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 4c42cd6a08dfd848a6e7ca94fad32a2a 2023-02-13 12:26:43,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for c9192eaa901c4d8dc716af4d533a03e9 2023-02-13 12:26:43,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. after waiting 0 ms 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for a3b734d5c30952e348ac86e3bc696201 2023-02-13 12:26:43,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 070d047471747182c749aaedda740bba 2023-02-13 12:26:43,446 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 976a16fc4c9a9b309025ba2a32e7ab08 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 956d36d3510f1e7466a907210667b173 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for da1f57ab313c133ba31c2c29865561b6 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 5a064ab3d6bc91c5da44588fa6275765 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 5d424be18d07d0a984d49d2450dc4b7e 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:26:43,447 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:43,447 DEBUG [RS:0;jenkins-hbase16:38471] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68aa53fc to 127.0.0.1:64508 2023-02-13 12:26:43,447 DEBUG [RS:0;jenkins-hbase16:38471] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:43,448 INFO [RS:0;jenkins-hbase16:38471] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 12:26:43,448 INFO [RS:0;jenkins-hbase16:38471] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 12:26:43,449 INFO [RS:0;jenkins-hbase16:38471] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 12:26:43,449 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 12:26:43,449 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-13 12:26:43,449 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 12:26:43,450 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 12:26:43,450 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 12:26:43,450 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 12:26:43,450 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 12:26:43,450 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.62 KB heapSize=78.23 KB 2023-02-13 12:26:43,451 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1505): Waiting on 070d047471747182c749aaedda740bba, 0b6d53fe273287f42b700182304d00f6, 1588230740, 31a6bfc09681f3a11506a4775f610e4d, 4993132286c530710e08273860fde1c5, 4b661928fda9be3aac0c0235707673ac, 4c42cd6a08dfd848a6e7ca94fad32a2a, 534c6c8744730b7c32dbd5f29d42ddff, 5a064ab3d6bc91c5da44588fa6275765, 5d424be18d07d0a984d49d2450dc4b7e, 73394efe2c0767e17a09a439425afd89, 956d36d3510f1e7466a907210667b173, 976a16fc4c9a9b309025ba2a32e7ab08, a3b734d5c30952e348ac86e3bc696201, adac9b9ed7bc6febc3aa79d4b3cce9dd, be0d091bd879acd2c49cb81bc7c86ecc, c410f61e2e0cf5f28a4b5c4a521e8eb0, c9192eaa901c4d8dc716af4d533a03e9, da1f57ab313c133ba31c2c29865561b6, e7ec1481802597d098b27d5a31dd86f7, e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:26:43,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/31a6bfc09681f3a11506a4775f610e4d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 12:26:43,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 31a6bfc09681f3a11506a4775f610e4d: Waiting for close lock at 1676291203445Running coprocessor pre-close hooks at 1676291203445Disabling compacts and flushes for region at 1676291203445Disabling writes for close at 1676291203446 (+1 ms)Writing region close event to WAL at 1676291203449 (+3 ms)Running coprocessor post-close hooks at 1676291203453 (+4 ms)Closed at 1676291203453 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1676291119959.31a6bfc09681f3a11506a4775f610e4d. 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing adac9b9ed7bc6febc3aa79d4b3cce9dd, disabling compactions & flushes 2023-02-13 12:26:43,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. after waiting 0 ms 2023-02-13 12:26:43,454 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:43,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/adac9b9ed7bc6febc3aa79d4b3cce9dd/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for adac9b9ed7bc6febc3aa79d4b3cce9dd: Waiting for close lock at 1676291203454Running coprocessor pre-close hooks at 1676291203454Disabling compacts and flushes for region at 1676291203454Disabling writes for close at 1676291203454Writing region close event to WAL at 1676291203454Running coprocessor post-close hooks at 1676291203458 (+4 ms)Closed at 1676291203459 (+1 ms) 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1676291181001.adac9b9ed7bc6febc3aa79d4b3cce9dd. 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 4993132286c530710e08273860fde1c5, disabling compactions & flushes 2023-02-13 12:26:43,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. after waiting 0 ms 2023-02-13 12:26:43,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:43,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/4993132286c530710e08273860fde1c5/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 4993132286c530710e08273860fde1c5: Waiting for close lock at 1676291203459Running coprocessor pre-close hooks at 1676291203459Disabling compacts and flushes for region at 1676291203459Disabling writes for close at 1676291203459Writing region close event to WAL at 1676291203460 (+1 ms)Running coprocessor post-close hooks at 1676291203463 (+3 ms)Closed at 1676291203463 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1676291149320.4993132286c530710e08273860fde1c5. 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 534c6c8744730b7c32dbd5f29d42ddff, disabling compactions & flushes 2023-02-13 12:26:43,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. after waiting 0 ms 2023-02-13 12:26:43,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:26:43,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/534c6c8744730b7c32dbd5f29d42ddff/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 12:26:43,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:26:43,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 534c6c8744730b7c32dbd5f29d42ddff: Waiting for close lock at 1676291203463Running coprocessor pre-close hooks at 1676291203463Disabling compacts and flushes for region at 1676291203463Disabling writes for close at 1676291203463Writing region close event to WAL at 1676291203464 (+1 ms)Running coprocessor post-close hooks at 1676291203468 (+4 ms)Closed at 1676291203468 2023-02-13 12:26:43,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1676291119959.534c6c8744730b7c32dbd5f29d42ddff. 2023-02-13 12:26:43,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing be0d091bd879acd2c49cb81bc7c86ecc, disabling compactions & flushes 2023-02-13 12:26:43,469 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:43,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:43,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. after waiting 0 ms 2023-02-13 12:26:43,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:43,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/be0d091bd879acd2c49cb81bc7c86ecc/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for be0d091bd879acd2c49cb81bc7c86ecc: Waiting for close lock at 1676291203469Running coprocessor pre-close hooks at 1676291203469Disabling compacts and flushes for region at 1676291203469Disabling writes for close at 1676291203469Writing region close event to WAL at 1676291203470 (+1 ms)Running coprocessor post-close hooks at 1676291203472 (+2 ms)Closed at 1676291203473 (+1 ms) 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1676291149320.be0d091bd879acd2c49cb81bc7c86ecc. 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 0b6d53fe273287f42b700182304d00f6, disabling compactions & flushes 2023-02-13 12:26:43,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. after waiting 0 ms 2023-02-13 12:26:43,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:26:43,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithWeeklyPolicy/0b6d53fe273287f42b700182304d00f6/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 12:26:43,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 0b6d53fe273287f42b700182304d00f6: Waiting for close lock at 1676291203473Running coprocessor pre-close hooks at 1676291203473Disabling compacts and flushes for region at 1676291203473Disabling writes for close at 1676291203473Writing region close event to WAL at 1676291203474 (+1 ms)Running coprocessor post-close hooks at 1676291203477 (+3 ms)Closed at 1676291203478 (+1 ms) 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1676291044661.0b6d53fe273287f42b700182304d00f6. 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e7ec1481802597d098b27d5a31dd86f7, disabling compactions & flushes 2023-02-13 12:26:43,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. after waiting 0 ms 2023-02-13 12:26:43,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:26:43,478 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing e7ec1481802597d098b27d5a31dd86f7 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-13 12:26:43,500 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:26:43,519 INFO [regionserver/jenkins-hbase16:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-13 12:26:43,651 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1505): Waiting on 070d047471747182c749aaedda740bba, 1588230740, 4b661928fda9be3aac0c0235707673ac, 4c42cd6a08dfd848a6e7ca94fad32a2a, 5a064ab3d6bc91c5da44588fa6275765, 5d424be18d07d0a984d49d2450dc4b7e, 73394efe2c0767e17a09a439425afd89, 956d36d3510f1e7466a907210667b173, 976a16fc4c9a9b309025ba2a32e7ab08, a3b734d5c30952e348ac86e3bc696201, c410f61e2e0cf5f28a4b5c4a521e8eb0, c9192eaa901c4d8dc716af4d533a03e9, da1f57ab313c133ba31c2c29865561b6, e7ec1481802597d098b27d5a31dd86f7, e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:26:43,852 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1505): Waiting on 070d047471747182c749aaedda740bba, 1588230740, 4b661928fda9be3aac0c0235707673ac, 4c42cd6a08dfd848a6e7ca94fad32a2a, 5a064ab3d6bc91c5da44588fa6275765, 5d424be18d07d0a984d49d2450dc4b7e, 73394efe2c0767e17a09a439425afd89, 956d36d3510f1e7466a907210667b173, 976a16fc4c9a9b309025ba2a32e7ab08, a3b734d5c30952e348ac86e3bc696201, c410f61e2e0cf5f28a4b5c4a521e8eb0, c9192eaa901c4d8dc716af4d533a03e9, da1f57ab313c133ba31c2c29865561b6, e7ec1481802597d098b27d5a31dd86f7, e8a189e2b6c4b0348126d69cf6d787a1 2023-02-13 12:26:43,861 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.34 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/info/17140b57cd8a40e19c0e45cdd0a86d34 2023-02-13 12:26:43,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/.tmp/info/9d9ceb53261a44e2a30dc9bc3e783c26 2023-02-13 12:26:43,885 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17140b57cd8a40e19c0e45cdd0a86d34 2023-02-13 12:26:43,897 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/.tmp/info/9d9ceb53261a44e2a30dc9bc3e783c26 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/info/9d9ceb53261a44e2a30dc9bc3e783c26 2023-02-13 12:26:43,897 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/rep_barrier/f055c5ee1bf3493bb735e6217b49bf6d 2023-02-13 12:26:43,901 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f055c5ee1bf3493bb735e6217b49bf6d 2023-02-13 12:26:43,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/info/9d9ceb53261a44e2a30dc9bc3e783c26, entries=3, sequenceid=7, filesize=4.8 K 2023-02-13 12:26:43,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for e7ec1481802597d098b27d5a31dd86f7 in 424ms, sequenceid=7, compaction requested=false 2023-02-13 12:26:43,905 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/namespace/e7ec1481802597d098b27d5a31dd86f7/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-13 12:26:43,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e7ec1481802597d098b27d5a31dd86f7: Waiting for close lock at 1676291203478Running coprocessor pre-close hooks at 1676291203478Disabling compacts and flushes for region at 1676291203478Disabling writes for close at 1676291203478Obtaining lock to block concurrent updates at 1676291203478Preparing flush snapshotting stores in e7ec1481802597d098b27d5a31dd86f7 at 1676291203478Finished memstore snapshotting hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1676291203478Flushing stores of hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. at 1676291203479 (+1 ms)Flushing e7ec1481802597d098b27d5a31dd86f7/info: creating writer at 1676291203479Flushing e7ec1481802597d098b27d5a31dd86f7/info: appending metadata at 1676291203481 (+2 ms)Flushing e7ec1481802597d098b27d5a31dd86f7/info: closing flushed file at 1676291203481Flushing e7ec1481802597d098b27d5a31dd86f7/info: reopening flushed file at 1676291203898 (+417 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for e7ec1481802597d098b27d5a31dd86f7 in 424ms, sequenceid=7, compaction requested=false at 1676291203902 (+4 ms)Writing region close event to WAL at 1676291203903 (+1 ms)Running coprocessor post-close hooks at 1676291203905 (+2 ms)Closed at 1676291203906 (+1 ms) 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676291014797.e7ec1481802597d098b27d5a31dd86f7. 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 73394efe2c0767e17a09a439425afd89, disabling compactions & flushes 2023-02-13 12:26:43,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. after waiting 0 ms 2023-02-13 12:26:43,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:26:43,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyPolicy/73394efe2c0767e17a09a439425afd89/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-13 12:26:43,909 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 73394efe2c0767e17a09a439425afd89: Waiting for close lock at 1676291203906Running coprocessor pre-close hooks at 1676291203906Disabling compacts and flushes for region at 1676291203906Disabling writes for close at 1676291203906Writing region close event to WAL at 1676291203907 (+1 ms)Running coprocessor post-close hooks at 1676291203909 (+2 ms)Closed at 1676291203909 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1676291064067.73394efe2c0767e17a09a439425afd89. 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 4b661928fda9be3aac0c0235707673ac, disabling compactions & flushes 2023-02-13 12:26:43,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. after waiting 0 ms 2023-02-13 12:26:43,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:43,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/4b661928fda9be3aac0c0235707673ac/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 4b661928fda9be3aac0c0235707673ac: Waiting for close lock at 1676291203910Running coprocessor pre-close hooks at 1676291203910Disabling compacts and flushes for region at 1676291203910Disabling writes for close at 1676291203910Writing region close event to WAL at 1676291203910Running coprocessor post-close hooks at 1676291203913 (+3 ms)Closed at 1676291203913 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1676291181001.4b661928fda9be3aac0c0235707673ac. 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing c410f61e2e0cf5f28a4b5c4a521e8eb0, disabling compactions & flushes 2023-02-13 12:26:43,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. after waiting 0 ms 2023-02-13 12:26:43,914 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:43,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/ns/testMinorCompaction/c410f61e2e0cf5f28a4b5c4a521e8eb0/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for c410f61e2e0cf5f28a4b5c4a521e8eb0: Waiting for close lock at 1676291203914Running coprocessor pre-close hooks at 1676291203914Disabling compacts and flushes for region at 1676291203914Disabling writes for close at 1676291203914Writing region close event to WAL at 1676291203916 (+2 ms)Running coprocessor post-close hooks at 1676291203919 (+3 ms)Closed at 1676291203919 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1676291149320.c410f61e2e0cf5f28a4b5c4a521e8eb0. 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 4c42cd6a08dfd848a6e7ca94fad32a2a, disabling compactions & flushes 2023-02-13 12:26:43,919 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. after waiting 0 ms 2023-02-13 12:26:43,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:26:43,922 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/4c42cd6a08dfd848a6e7ca94fad32a2a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-13 12:26:43,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 4c42cd6a08dfd848a6e7ca94fad32a2a: Waiting for close lock at 1676291203919Running coprocessor pre-close hooks at 1676291203919Disabling compacts and flushes for region at 1676291203919Disabling writes for close at 1676291203919Writing region close event to WAL at 1676291203920 (+1 ms)Running coprocessor post-close hooks at 1676291203922 (+2 ms)Closed at 1676291203923 (+1 ms) 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1676291058797.4c42cd6a08dfd848a6e7ca94fad32a2a. 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing c9192eaa901c4d8dc716af4d533a03e9, disabling compactions & flushes 2023-02-13 12:26:43,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. after waiting 0 ms 2023-02-13 12:26:43,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:43,929 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMinorCompactionWithMonthlyPolicy/c9192eaa901c4d8dc716af4d533a03e9/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 12:26:43,930 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for c9192eaa901c4d8dc716af4d533a03e9: Waiting for close lock at 1676291203923Running coprocessor pre-close hooks at 1676291203923Disabling compacts and flushes for region at 1676291203923Disabling writes for close at 1676291203923Writing region close event to WAL at 1676291203925 (+2 ms)Running coprocessor post-close hooks at 1676291203929 (+4 ms)Closed at 1676291203930 (+1 ms) 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1676291169026.c9192eaa901c4d8dc716af4d533a03e9. 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing a3b734d5c30952e348ac86e3bc696201, disabling compactions & flushes 2023-02-13 12:26:43,930 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. after waiting 0 ms 2023-02-13 12:26:43,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:26:43,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a3b734d5c30952e348ac86e3bc696201/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 12:26:43,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for a3b734d5c30952e348ac86e3bc696201: Waiting for close lock at 1676291203930Running coprocessor pre-close hooks at 1676291203930Disabling compacts and flushes for region at 1676291203930Disabling writes for close at 1676291203930Writing region close event to WAL at 1676291203931 (+1 ms)Running coprocessor post-close hooks at 1676291203933 (+2 ms)Closed at 1676291203934 (+1 ms) 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676291078562.a3b734d5c30952e348ac86e3bc696201. 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 070d047471747182c749aaedda740bba, disabling compactions & flushes 2023-02-13 12:26:43,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. after waiting 0 ms 2023-02-13 12:26:43,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:43,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionFromAdmin/070d047471747182c749aaedda740bba/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-13 12:26:43,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 070d047471747182c749aaedda740bba: Waiting for close lock at 1676291203934Running coprocessor pre-close hooks at 1676291203934Disabling compacts and flushes for region at 1676291203934Disabling writes for close at 1676291203934Writing region close event to WAL at 1676291203935 (+1 ms)Running coprocessor post-close hooks at 1676291203938 (+3 ms)Closed at 1676291203938 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1676291181001.070d047471747182c749aaedda740bba. 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 976a16fc4c9a9b309025ba2a32e7ab08, disabling compactions & flushes 2023-02-13 12:26:43,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. after waiting 0 ms 2023-02-13 12:26:43,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:26:43,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/976a16fc4c9a9b309025ba2a32e7ab08/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 12:26:43,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 976a16fc4c9a9b309025ba2a32e7ab08: Waiting for close lock at 1676291203938Running coprocessor pre-close hooks at 1676291203938Disabling compacts and flushes for region at 1676291203938Disabling writes for close at 1676291203938Writing region close event to WAL at 1676291203938Running coprocessor post-close hooks at 1676291203941 (+3 ms)Closed at 1676291203941 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1676291058797.976a16fc4c9a9b309025ba2a32e7ab08. 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 956d36d3510f1e7466a907210667b173, disabling compactions & flushes 2023-02-13 12:26:43,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. after waiting 0 ms 2023-02-13 12:26:43,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:26:43,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testScannerOnBulkLoadRefHFiles/956d36d3510f1e7466a907210667b173/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-13 12:26:43,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 956d36d3510f1e7466a907210667b173: Waiting for close lock at 1676291203941Running coprocessor pre-close hooks at 1676291203941Disabling compacts and flushes for region at 1676291203941Disabling writes for close at 1676291203941Writing region close event to WAL at 1676291203941Running coprocessor post-close hooks at 1676291203944 (+3 ms)Closed at 1676291203944 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1676291058797.956d36d3510f1e7466a907210667b173. 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing da1f57ab313c133ba31c2c29865561b6, disabling compactions & flushes 2023-02-13 12:26:43,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. after waiting 0 ms 2023-02-13 12:26:43,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:26:43,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/da1f57ab313c133ba31c2c29865561b6/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-02-13 12:26:43,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:26:43,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for da1f57ab313c133ba31c2c29865561b6: Waiting for close lock at 1676291203944Running coprocessor pre-close hooks at 1676291203944Disabling compacts and flushes for region at 1676291203944Disabling writes for close at 1676291203944Writing region close event to WAL at 1676291203945 (+1 ms)Running coprocessor post-close hooks at 1676291203947 (+2 ms)Closed at 1676291203947 2023-02-13 12:26:43,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676291016189.da1f57ab313c133ba31c2c29865561b6. 2023-02-13 12:26:43,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5a064ab3d6bc91c5da44588fa6275765, disabling compactions & flushes 2023-02-13 12:26:43,948 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:26:43,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:26:43,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. after waiting 0 ms 2023-02-13 12:26:43,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:26:43,950 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithMonthlyPolicy/5a064ab3d6bc91c5da44588fa6275765/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-13 12:26:43,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5a064ab3d6bc91c5da44588fa6275765: Waiting for close lock at 1676291203948Running coprocessor pre-close hooks at 1676291203948Disabling compacts and flushes for region at 1676291203948Disabling writes for close at 1676291203948Writing region close event to WAL at 1676291203948Running coprocessor post-close hooks at 1676291203951 (+3 ms)Closed at 1676291203951 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1676291029414.5a064ab3d6bc91c5da44588fa6275765. 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5d424be18d07d0a984d49d2450dc4b7e, disabling compactions & flushes 2023-02-13 12:26:43,951 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. after waiting 0 ms 2023-02-13 12:26:43,951 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testCompactionWithHFileLink/5d424be18d07d0a984d49d2450dc4b7e/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-13 12:26:43,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5d424be18d07d0a984d49d2450dc4b7e: Waiting for close lock at 1676291203951Running coprocessor pre-close hooks at 1676291203951Disabling compacts and flushes for region at 1676291203951Disabling writes for close at 1676291203951Writing region close event to WAL at 1676291203952 (+1 ms)Running coprocessor post-close hooks at 1676291203954 (+2 ms)Closed at 1676291203954 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1676291119959.5d424be18d07d0a984d49d2450dc4b7e. 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e8a189e2b6c4b0348126d69cf6d787a1, disabling compactions & flushes 2023-02-13 12:26:43,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. after waiting 0 ms 2023-02-13 12:26:43,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:26:43,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/e8a189e2b6c4b0348126d69cf6d787a1/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-13 12:26:43,957 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:26:43,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e8a189e2b6c4b0348126d69cf6d787a1: Waiting for close lock at 1676291203954Running coprocessor pre-close hooks at 1676291203954Disabling compacts and flushes for region at 1676291203954Disabling writes for close at 1676291203954Writing region close event to WAL at 1676291203955 (+1 ms)Running coprocessor post-close hooks at 1676291203957 (+2 ms)Closed at 1676291203957 2023-02-13 12:26:43,957 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676291095591.e8a189e2b6c4b0348126d69cf6d787a1. 2023-02-13 12:26:44,052 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 12:26:44,253 DEBUG [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 12:26:44,309 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/table/5fb664c01c7d4de7a8a0933311dbf7fb 2023-02-13 12:26:44,321 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fb664c01c7d4de7a8a0933311dbf7fb 2023-02-13 12:26:44,323 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/info/17140b57cd8a40e19c0e45cdd0a86d34 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/info/17140b57cd8a40e19c0e45cdd0a86d34 2023-02-13 12:26:44,329 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17140b57cd8a40e19c0e45cdd0a86d34 2023-02-13 12:26:44,329 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/info/17140b57cd8a40e19c0e45cdd0a86d34, entries=226, sequenceid=130, filesize=37.9 K 2023-02-13 12:26:44,330 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/rep_barrier/f055c5ee1bf3493bb735e6217b49bf6d as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/rep_barrier/f055c5ee1bf3493bb735e6217b49bf6d 2023-02-13 12:26:44,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f055c5ee1bf3493bb735e6217b49bf6d 2023-02-13 12:26:44,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/rep_barrier/f055c5ee1bf3493bb735e6217b49bf6d, entries=1, sequenceid=130, filesize=5.0 K 2023-02-13 12:26:44,338 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/.tmp/table/5fb664c01c7d4de7a8a0933311dbf7fb as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/table/5fb664c01c7d4de7a8a0933311dbf7fb 2023-02-13 12:26:44,341 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5fb664c01c7d4de7a8a0933311dbf7fb 2023-02-13 12:26:44,342 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/table/5fb664c01c7d4de7a8a0933311dbf7fb, entries=27, sequenceid=130, filesize=6.6 K 2023-02-13 12:26:44,344 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.62 KB/51839, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 894ms, sequenceid=130, compaction requested=false 2023-02-13 12:26:44,348 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-13 12:26:44,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 12:26:44,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 12:26:44,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676291203449Running coprocessor pre-close hooks at 1676291203449Disabling compacts and flushes for region at 1676291203449Disabling writes for close at 1676291203450 (+1 ms)Obtaining lock to block concurrent updates at 1676291203450Preparing flush snapshotting stores in 1588230740 at 1676291203450Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51839, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1676291203451 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676291203451Flushing 1588230740/info: creating writer at 1676291203451Flushing 1588230740/info: appending metadata at 1676291203455 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676291203455Flushing 1588230740/rep_barrier: creating writer at 1676291203886 (+431 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676291203888 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676291203888Flushing 1588230740/table: creating writer at 1676291203901 (+13 ms)Flushing 1588230740/table: appending metadata at 1676291203904 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676291203904Flushing 1588230740/info: reopening flushed file at 1676291204323 (+419 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676291204331 (+8 ms)Flushing 1588230740/table: reopening flushed file at 1676291204338 (+7 ms)Finished flush of dataSize ~50.62 KB/51839, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 894ms, sequenceid=130, compaction requested=false at 1676291204344 (+6 ms)Writing region close event to WAL at 1676291204345 (+1 ms)Running coprocessor post-close hooks at 1676291204349 (+4 ms)Closed at 1676291204349 2023-02-13 12:26:44,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 12:26:44,424 INFO [regionserver/jenkins-hbase16:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 12:26:44,424 INFO [regionserver/jenkins-hbase16:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 12:26:44,453 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,38471,1676291011293; all regions closed. 2023-02-13 12:26:44,466 DEBUG [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/oldWALs 2023-02-13 12:26:44,466 INFO [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C38471%2C1676291011293.meta:.meta(num 1676291014530) 2023-02-13 12:26:44,472 DEBUG [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/oldWALs 2023-02-13 12:26:44,472 INFO [RS:0;jenkins-hbase16:38471] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C38471%2C1676291011293:(num 1676291013513) 2023-02-13 12:26:44,472 DEBUG [RS:0;jenkins-hbase16:38471] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:44,472 INFO [RS:0;jenkins-hbase16:38471] regionserver.LeaseManager(133): Closed leases 2023-02-13 12:26:44,473 INFO [RS:0;jenkins-hbase16:38471] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 12:26:44,473 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:26:44,474 INFO [RS:0;jenkins-hbase16:38471] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:38471 2023-02-13 12:26:44,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,38471,1676291011293 2023-02-13 12:26:44,515 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:26:44,516 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@22ca1a3d rejected from java.util.concurrent.ThreadPoolExecutor@26f59abc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:44,516 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 12:26:44,516 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@20b70e25 rejected from java.util.concurrent.ThreadPoolExecutor@26f59abc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:44,524 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,38471,1676291011293] 2023-02-13 12:26:44,524 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,38471,1676291011293; numProcessing=1 2023-02-13 12:26:44,532 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,38471,1676291011293 already deleted, retry=false 2023-02-13 12:26:44,532 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,38471,1676291011293 expired; onlineServers=0 2023-02-13 12:26:44,532 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,41493,1676291010224' ***** 2023-02-13 12:26:44,532 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 12:26:44,533 DEBUG [M:0;jenkins-hbase16:41493] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e36ced1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 12:26:44,533 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,41493,1676291010224 2023-02-13 12:26:44,533 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,41493,1676291010224; all regions closed. 2023-02-13 12:26:44,533 DEBUG [M:0;jenkins-hbase16:41493] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 12:26:44,533 DEBUG [M:0;jenkins-hbase16:41493] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 12:26:44,534 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 12:26:44,534 DEBUG [M:0;jenkins-hbase16:41493] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 12:26:44,534 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676291013249] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676291013249,5,FailOnTimeoutGroup] 2023-02-13 12:26:44,534 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676291013250] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676291013250,5,FailOnTimeoutGroup] 2023-02-13 12:26:44,534 INFO [M:0;jenkins-hbase16:41493] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 12:26:44,534 INFO [M:0;jenkins-hbase16:41493] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 12:26:44,534 INFO [M:0;jenkins-hbase16:41493] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-02-13 12:26:44,535 DEBUG [M:0;jenkins-hbase16:41493] master.HMaster(1502): Stopping service threads 2023-02-13 12:26:44,535 INFO [M:0;jenkins-hbase16:41493] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 12:26:44,536 ERROR [M:0;jenkins-hbase16:41493] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-13 12:26:44,537 INFO [M:0;jenkins-hbase16:41493] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 12:26:44,537 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 12:26:44,540 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 12:26:44,541 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 12:26:44,541 DEBUG [M:0;jenkins-hbase16:41493] zookeeper.ZKUtil(398): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 12:26:44,541 WARN [M:0;jenkins-hbase16:41493] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 12:26:44,541 INFO [M:0;jenkins-hbase16:41493] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 12:26:44,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 12:26:44,541 INFO [M:0;jenkins-hbase16:41493] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 12:26:44,542 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 12:26:44,542 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:26:44,542 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:26:44,542 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 12:26:44,542 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:26:44,542 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=513.51 KB heapSize=599.63 KB 2023-02-13 12:26:44,624 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:26:44,624 INFO [RS:0;jenkins-hbase16:38471] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,38471,1676291011293; zookeeper connection closed. 2023-02-13 12:26:44,624 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5874b361 rejected from java.util.concurrent.ThreadPoolExecutor@26f59abc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:44,625 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): regionserver:38471-0x10052d859b90001, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:26:44,625 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@66f5432d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@66f5432d 2023-02-13 12:26:44,625 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1b061f31 rejected from java.util.concurrent.ThreadPoolExecutor@26f59abc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:44,625 INFO [Listener at localhost.localdomain/33629] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-13 12:26:44,953 INFO [M:0;jenkins-hbase16:41493] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=513.51 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2625101ac8f947a4be60ff4662bdfd71 2023-02-13 12:26:44,962 INFO [M:0;jenkins-hbase16:41493] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2625101ac8f947a4be60ff4662bdfd71 2023-02-13 12:26:44,964 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2625101ac8f947a4be60ff4662bdfd71 as hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2625101ac8f947a4be60ff4662bdfd71 2023-02-13 12:26:44,968 INFO [M:0;jenkins-hbase16:41493] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2625101ac8f947a4be60ff4662bdfd71 2023-02-13 12:26:44,968 INFO [M:0;jenkins-hbase16:41493] regionserver.HStore(1080): Added hdfs://localhost.localdomain:39971/user/jenkins/test-data/2aaa731c-bc40-16a5-7738-8d8ad5e005a0/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2625101ac8f947a4be60ff4662bdfd71, entries=123, sequenceid=1011, filesize=33.3 K 2023-02-13 12:26:44,969 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegion(2947): Finished flush of dataSize ~513.51 KB/525836, heapSize ~599.62 KB/614008, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 427ms, sequenceid=1011, compaction requested=false 2023-02-13 12:26:44,970 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 12:26:44,970 DEBUG [M:0;jenkins-hbase16:41493] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676291204542Disabling compacts and flushes for region at 1676291204542Disabling writes for close at 1676291204542Obtaining lock to block concurrent updates at 1676291204542Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676291204542Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525836, getHeapSize=614008, getOffHeapSize=0, getCellsCount=1166 at 1676291204543 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676291204543Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676291204543Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676291204548 (+5 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676291204548Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676291204964 (+416 ms)Finished flush of dataSize ~513.51 KB/525836, heapSize ~599.62 KB/614008, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 427ms, sequenceid=1011, compaction requested=false at 1676291204969 (+5 ms)Writing region close event to WAL at 1676291204969Closed at 1676291204970 (+1 ms) 2023-02-13 12:26:44,973 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 12:26:44,973 INFO [M:0;jenkins-hbase16:41493] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 12:26:44,974 INFO [M:0;jenkins-hbase16:41493] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:41493 2023-02-13 12:26:44,982 DEBUG [M:0;jenkins-hbase16:41493] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,41493,1676291010224 already deleted, retry=false 2023-02-13 12:26:45,091 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:26:45,091 INFO [M:0;jenkins-hbase16:41493] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,41493,1676291010224; zookeeper connection closed. 2023-02-13 12:26:45,091 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1f1e0719 rejected from java.util.concurrent.ThreadPoolExecutor@31b279cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:45,091 DEBUG [Listener at localhost.localdomain/33629-EventThread] zookeeper.ZKWatcher(600): master:41493-0x10052d859b90000, quorum=127.0.0.1:64508, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 12:26:45,091 ERROR [Listener at localhost.localdomain/33629-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@67211b6a rejected from java.util.concurrent.ThreadPoolExecutor@31b279cd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 12:26:45,094 WARN [Listener at localhost.localdomain/33629] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 12:26:45,100 INFO [Listener at localhost.localdomain/33629] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 12:26:45,214 WARN [BP-878267689-188.40.62.62-1676291006386 heartbeating to localhost.localdomain/127.0.0.1:39971] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 12:26:45,214 WARN [BP-878267689-188.40.62.62-1676291006386 heartbeating to localhost.localdomain/127.0.0.1:39971] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-878267689-188.40.62.62-1676291006386 (Datanode Uuid 368b0532-ef68-4161-bbce-2cda1773f909) service to localhost.localdomain/127.0.0.1:39971 2023-02-13 12:26:45,218 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/dfs/data/data1/current/BP-878267689-188.40.62.62-1676291006386] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:26:45,219 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/08918f5c-b380-0893-f26a-202764f97a9d/cluster_17b78fbd-1102-f5dc-2590-adef2fb0bc71/dfs/data/data2/current/BP-878267689-188.40.62.62-1676291006386] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 12:26:45,241 INFO [Listener at localhost.localdomain/33629] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 12:26:45,379 INFO [Listener at localhost.localdomain/33629] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 12:26:45,482 INFO [Listener at localhost.localdomain/33629] hbase.HBaseTestingUtility(1293): Minicluster is down