2023-03-23 08:53:30,729 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-23 08:53:31,040 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071 2023-03-23 08:53:31,053 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-23 08:53:31,053 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2, deleteOnExit=true 2023-03-23 08:53:31,054 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-23 08:53:31,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/test.cache.data in system properties and HBase conf 2023-03-23 08:53:31,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/hadoop.tmp.dir in system properties and HBase conf 2023-03-23 08:53:31,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/hadoop.log.dir in system properties and HBase conf 2023-03-23 08:53:31,056 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-23 08:53:31,056 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-23 08:53:31,056 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-23 08:53:31,175 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-23 08:53:31,599 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-23 08:53:31,603 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-23 08:53:31,603 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-23 08:53:31,604 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-23 08:53:31,604 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-23 08:53:31,604 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-23 08:53:31,605 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-23 08:53:31,605 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-23 08:53:31,605 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-23 08:53:31,607 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-23 08:53:31,607 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/nfs.dump.dir in system properties and HBase conf 2023-03-23 08:53:31,608 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/java.io.tmpdir in system properties and HBase conf 2023-03-23 08:53:31,608 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-23 08:53:31,609 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-23 08:53:31,609 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-23 08:53:32,203 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-23 08:53:32,208 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-23 08:53:32,505 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-23 08:53:32,689 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-23 08:53:32,711 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-23 08:53:32,751 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-23 08:53:32,783 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/java.io.tmpdir/Jetty_localhost_34709_hdfs____72pnhl/webapp 2023-03-23 08:53:32,927 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34709 2023-03-23 08:53:32,943 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-23 08:53:32,944 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-23 08:53:33,499 WARN [Listener at localhost/42513] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-23 08:53:33,580 WARN [Listener at localhost/42513] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-23 08:53:33,604 WARN [Listener at localhost/42513] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-23 08:53:33,612 INFO [Listener at localhost/42513] log.Slf4jLog(67): jetty-6.1.26 2023-03-23 08:53:33,618 INFO [Listener at localhost/42513] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/java.io.tmpdir/Jetty_localhost_37393_datanode____.cbnd2o/webapp 2023-03-23 08:53:33,752 INFO [Listener at localhost/42513] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37393 2023-03-23 08:53:34,165 WARN [Listener at localhost/38371] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-23 08:53:34,694 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9620e3a680975b77: Processing first storage report for DS-13c6e318-1914-4f25-9010-900597ea756c from datanode 53bda90c-65e6-4972-9c12-411d3cabc1c9 2023-03-23 08:53:34,696 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9620e3a680975b77: from storage DS-13c6e318-1914-4f25-9010-900597ea756c node DatanodeRegistration(127.0.0.1:44693, datanodeUuid=53bda90c-65e6-4972-9c12-411d3cabc1c9, infoPort=40119, infoSecurePort=0, ipcPort=38371, storageInfo=lv=-57;cid=testClusterID;nsid=1040421460;c=1679561612282), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-23 08:53:34,696 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9620e3a680975b77: Processing first storage report for DS-7af387b5-5268-4af9-934f-397028337bf1 from datanode 53bda90c-65e6-4972-9c12-411d3cabc1c9 2023-03-23 08:53:34,696 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9620e3a680975b77: from storage DS-7af387b5-5268-4af9-934f-397028337bf1 node DatanodeRegistration(127.0.0.1:44693, datanodeUuid=53bda90c-65e6-4972-9c12-411d3cabc1c9, infoPort=40119, infoSecurePort=0, ipcPort=38371, storageInfo=lv=-57;cid=testClusterID;nsid=1040421460;c=1679561612282), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-23 08:53:34,760 DEBUG [Listener at localhost/38371] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071 2023-03-23 08:53:34,824 INFO [Listener at localhost/38371] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/zookeeper_0, clientPort=50031, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-23 08:53:34,841 INFO [Listener at localhost/38371] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=50031 2023-03-23 08:53:34,851 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:34,853 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:35,520 INFO [Listener at localhost/38371] util.FSUtils(471): Created version file at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376 with version=8 2023-03-23 08:53:35,520 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/hbase-staging 2023-03-23 08:53:35,850 INFO [Listener at localhost/38371] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-23 08:53:36,368 INFO [Listener at localhost/38371] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-23 08:53:36,400 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,401 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,401 INFO [Listener at localhost/38371] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-23 08:53:36,401 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,401 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-23 08:53:36,546 INFO [Listener at localhost/38371] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-23 08:53:36,623 DEBUG [Listener at localhost/38371] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-23 08:53:36,722 INFO [Listener at localhost/38371] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37881 2023-03-23 08:53:36,737 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:36,739 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:36,765 INFO [Listener at localhost/38371] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37881 connecting to ZooKeeper ensemble=127.0.0.1:50031 2023-03-23 08:53:36,808 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:378810x0, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-23 08:53:36,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37881-0x10110944f9e0000 connected 2023-03-23 08:53:36,845 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-23 08:53:36,846 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-23 08:53:36,849 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-23 08:53:36,857 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37881 2023-03-23 08:53:36,857 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37881 2023-03-23 08:53:36,858 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37881 2023-03-23 08:53:36,858 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37881 2023-03-23 08:53:36,859 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37881 2023-03-23 08:53:36,865 INFO [Listener at localhost/38371] master.HMaster(444): hbase.rootdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376, hbase.cluster.distributed=false 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-23 08:53:36,942 INFO [Listener at localhost/38371] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-23 08:53:36,947 INFO [Listener at localhost/38371] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-23 08:53:36,950 INFO [Listener at localhost/38371] ipc.NettyRpcServer(120): Bind to /172.31.14.131:32923 2023-03-23 08:53:36,952 INFO [Listener at localhost/38371] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-23 08:53:36,961 DEBUG [Listener at localhost/38371] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-23 08:53:36,962 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:36,965 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:36,967 INFO [Listener at localhost/38371] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:32923 connecting to ZooKeeper ensemble=127.0.0.1:50031 2023-03-23 08:53:36,970 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:329230x0, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-23 08:53:36,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:32923-0x10110944f9e0001 connected 2023-03-23 08:53:36,971 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-23 08:53:36,973 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-23 08:53:36,973 DEBUG [Listener at localhost/38371] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-23 08:53:36,974 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=32923 2023-03-23 08:53:36,975 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=32923 2023-03-23 08:53:36,975 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=32923 2023-03-23 08:53:36,975 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=32923 2023-03-23 08:53:36,976 DEBUG [Listener at localhost/38371] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=32923 2023-03-23 08:53:36,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:36,987 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-23 08:53:36,988 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:37,007 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-23 08:53:37,007 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-23 08:53:37,007 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:37,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-23 08:53:37,009 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37881,1679561615669 from backup master directory 2023-03-23 08:53:37,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-23 08:53:37,014 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:37,014 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-23 08:53:37,015 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-23 08:53:37,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:37,019 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-23 08:53:37,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-23 08:53:37,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/hbase.id with ID: 3c4c1bfe-7438-455b-bdde-47f70826337b 2023-03-23 08:53:37,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:37,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:37,216 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x16dda0de to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:37,250 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73dfbdf4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:37,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-23 08:53:37,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-23 08:53:37,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-23 08:53:37,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-23 08:53:37,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-23 08:53:37,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-23 08:53:37,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-23 08:53:37,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store-tmp 2023-03-23 08:53:37,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:37,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-23 08:53:37,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:53:37,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:53:37,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-23 08:53:37,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:53:37,768 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:53:37,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-23 08:53:37,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/WALs/jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:37,790 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37881%2C1679561615669, suffix=, logDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/WALs/jenkins-hbase4.apache.org,37881,1679561615669, archiveDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/oldWALs, maxLogs=10 2023-03-23 08:53:37,851 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK] 2023-03-23 08:53:37,861 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-23 08:53:37,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/WALs/jenkins-hbase4.apache.org,37881,1679561615669/jenkins-hbase4.apache.org%2C37881%2C1679561615669.1679561617802 2023-03-23 08:53:37,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK]] 2023-03-23 08:53:37,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:37,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:37,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:37,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:38,015 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:38,024 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-23 08:53:38,056 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-23 08:53:38,071 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:38,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:38,081 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:38,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-23 08:53:38,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:53:38,106 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73658798, jitterRate=0.09760162234306335}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:53:38,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-23 08:53:38,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-23 08:53:38,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-23 08:53:38,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-23 08:53:38,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-23 08:53:38,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-23 08:53:38,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-03-23 08:53:38,164 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-23 08:53:38,188 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-23 08:53:38,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-23 08:53:38,223 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-23 08:53:38,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-23 08:53:38,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-23 08:53:38,236 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-23 08:53:38,241 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-23 08:53:38,244 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:38,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-23 08:53:38,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-23 08:53:38,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-23 08:53:38,267 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-23 08:53:38,267 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-23 08:53:38,268 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:38,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37881,1679561615669, sessionid=0x10110944f9e0000, setting cluster-up flag (Was=false) 2023-03-23 08:53:38,283 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:38,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-23 08:53:38,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:38,296 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:38,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-23 08:53:38,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:38,306 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp 2023-03-23 08:53:38,381 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(952): ClusterId : 3c4c1bfe-7438-455b-bdde-47f70826337b 2023-03-23 08:53:38,386 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-23 08:53:38,392 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-23 08:53:38,392 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-23 08:53:38,399 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-23 08:53:38,401 DEBUG [RS:0;jenkins-hbase4:32923] zookeeper.ReadOnlyZKClient(139): Connect 0x66db461f to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:38,410 DEBUG [RS:0;jenkins-hbase4:32923] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@681ca4b4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:38,411 DEBUG [RS:0;jenkins-hbase4:32923] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c232d6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-23 08:53:38,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-23 08:53:38,438 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:32923 2023-03-23 08:53:38,443 INFO [RS:0;jenkins-hbase4:32923] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-23 08:53:38,443 INFO [RS:0;jenkins-hbase4:32923] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-23 08:53:38,443 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1023): About to register with Master. 2023-03-23 08:53:38,446 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37881,1679561615669 with isa=jenkins-hbase4.apache.org/172.31.14.131:32923, startcode=1679561616941 2023-03-23 08:53:38,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-23 08:53:38,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-23 08:53:38,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-23 08:53:38,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-23 08:53:38,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-23 08:53:38,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-23 08:53:38,452 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,457 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1679561648456 2023-03-23 08:53:38,459 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-23 08:53:38,462 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-23 08:53:38,463 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-23 08:53:38,465 DEBUG [RS:0;jenkins-hbase4:32923] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-23 08:53:38,469 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-23 08:53:38,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-23 08:53:38,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-23 08:53:38,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-23 08:53:38,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-23 08:53:38,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-23 08:53:38,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-23 08:53:38,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-23 08:53:38,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-23 08:53:38,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-23 08:53:38,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-23 08:53:38,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679561618488,5,FailOnTimeoutGroup] 2023-03-23 08:53:38,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679561618489,5,FailOnTimeoutGroup] 2023-03-23 08:53:38,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,489 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-23 08:53:38,491 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,525 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38881, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-23 08:53:38,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,551 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376 2023-03-23 08:53:38,552 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:42513 2023-03-23 08:53:38,552 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-23 08:53:38,557 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-23 08:53:38,557 DEBUG [RS:0;jenkins-hbase4:32923] zookeeper.ZKUtil(162): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,558 WARN [RS:0;jenkins-hbase4:32923] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-23 08:53:38,558 INFO [RS:0;jenkins-hbase4:32923] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-23 08:53:38,559 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1947): logDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/WALs/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,560 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,32923,1679561616941] 2023-03-23 08:53:38,568 DEBUG [RS:0;jenkins-hbase4:32923] zookeeper.ZKUtil(162): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,578 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-23 08:53:38,587 INFO [RS:0;jenkins-hbase4:32923] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-23 08:53:38,605 INFO [RS:0;jenkins-hbase4:32923] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-23 08:53:38,610 INFO [RS:0;jenkins-hbase4:32923] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:53:38,610 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,612 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-23 08:53:38,619 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,620 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,620 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,621 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,621 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,621 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,621 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-23 08:53:38,621 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,622 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,622 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,622 DEBUG [RS:0;jenkins-hbase4:32923] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-23 08:53:38,623 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,623 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,623 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,639 INFO [RS:0;jenkins-hbase4:32923] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-23 08:53:38,642 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,32923,1679561616941-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:38,659 INFO [RS:0;jenkins-hbase4:32923] regionserver.Replication(203): jenkins-hbase4.apache.org,32923,1679561616941 started 2023-03-23 08:53:38,659 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,32923,1679561616941, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:32923, sessionid=0x10110944f9e0001 2023-03-23 08:53:38,660 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-23 08:53:38,660 DEBUG [RS:0;jenkins-hbase4:32923] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,660 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,32923,1679561616941' 2023-03-23 08:53:38,660 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,32923,1679561616941' 2023-03-23 08:53:38,661 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-23 08:53:38,662 DEBUG [RS:0;jenkins-hbase4:32923] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-23 08:53:38,662 DEBUG [RS:0;jenkins-hbase4:32923] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-23 08:53:38,662 INFO [RS:0;jenkins-hbase4:32923] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-23 08:53:38,662 INFO [RS:0;jenkins-hbase4:32923] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-23 08:53:38,774 INFO [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32923%2C1679561616941, suffix=, logDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/WALs/jenkins-hbase4.apache.org,32923,1679561616941, archiveDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/oldWALs, maxLogs=32 2023-03-23 08:53:38,797 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK] 2023-03-23 08:53:38,800 INFO [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/WALs/jenkins-hbase4.apache.org,32923,1679561616941/jenkins-hbase4.apache.org%2C32923%2C1679561616941.1679561618776 2023-03-23 08:53:38,801 DEBUG [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK]] 2023-03-23 08:53:38,904 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:38,906 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:38,906 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376 2023-03-23 08:53:39,320 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:39,323 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-23 08:53:39,326 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/info 2023-03-23 08:53:39,327 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-23 08:53:39,329 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,329 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-23 08:53:39,332 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/rep_barrier 2023-03-23 08:53:39,333 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-23 08:53:39,334 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,334 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-23 08:53:39,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/table 2023-03-23 08:53:39,337 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-23 08:53:39,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,339 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740 2023-03-23 08:53:39,340 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740 2023-03-23 08:53:39,343 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-23 08:53:39,345 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-23 08:53:39,348 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:53:39,349 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65420603, jitterRate=-0.025157049298286438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-23 08:53:39,349 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-23 08:53:39,349 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-23 08:53:39,349 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-23 08:53:39,349 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-23 08:53:39,349 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-23 08:53:39,349 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-23 08:53:39,350 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-23 08:53:39,350 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-23 08:53:39,356 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-23 08:53:39,356 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-23 08:53:39,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-23 08:53:39,375 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-23 08:53:39,377 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-23 08:53:39,529 DEBUG [jenkins-hbase4:37881] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-23 08:53:39,532 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,32923,1679561616941, state=OPENING 2023-03-23 08:53:39,540 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-23 08:53:39,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:39,543 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-23 08:53:39,545 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:39,732 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:39,735 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-23 08:53:39,739 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36150, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-23 08:53:39,752 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-23 08:53:39,752 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-23 08:53:39,756 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C32923%2C1679561616941.meta, suffix=.meta, logDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/WALs/jenkins-hbase4.apache.org,32923,1679561616941, archiveDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/oldWALs, maxLogs=32 2023-03-23 08:53:39,771 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK] 2023-03-23 08:53:39,776 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/WALs/jenkins-hbase4.apache.org,32923,1679561616941/jenkins-hbase4.apache.org%2C32923%2C1679561616941.meta.1679561619757.meta 2023-03-23 08:53:39,776 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44693,DS-13c6e318-1914-4f25-9010-900597ea756c,DISK]] 2023-03-23 08:53:39,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:39,779 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-23 08:53:39,797 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-23 08:53:39,802 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-23 08:53:39,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-23 08:53:39,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:39,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-23 08:53:39,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-23 08:53:39,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-23 08:53:39,813 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/info 2023-03-23 08:53:39,813 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/info 2023-03-23 08:53:39,814 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-23 08:53:39,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-23 08:53:39,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/rep_barrier 2023-03-23 08:53:39,817 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/rep_barrier 2023-03-23 08:53:39,817 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-23 08:53:39,818 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,818 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-23 08:53:39,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/table 2023-03-23 08:53:39,820 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/table 2023-03-23 08:53:39,820 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-23 08:53:39,821 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:39,823 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740 2023-03-23 08:53:39,826 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740 2023-03-23 08:53:39,830 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-23 08:53:39,832 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-23 08:53:39,834 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63283704, jitterRate=-0.0569993257522583}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-23 08:53:39,834 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-23 08:53:39,844 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1679561619724 2023-03-23 08:53:39,867 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-23 08:53:39,867 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-23 08:53:39,868 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,32923,1679561616941, state=OPEN 2023-03-23 08:53:39,871 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-23 08:53:39,871 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-23 08:53:39,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-23 08:53:39,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,32923,1679561616941 in 326 msec 2023-03-23 08:53:39,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-23 08:53:39,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 514 msec 2023-03-23 08:53:39,886 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5250 sec 2023-03-23 08:53:39,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1679561619886, completionTime=-1 2023-03-23 08:53:39,886 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-23 08:53:39,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-03-23 08:53:39,955 DEBUG [hconnection-0x1b0aee48-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:39,959 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36162, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:39,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-03-23 08:53:39,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1679561679977 2023-03-23 08:53:39,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1679561739977 2023-03-23 08:53:39,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 90 msec 2023-03-23 08:53:39,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37881,1679561615669-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:39,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37881,1679561615669-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:39,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37881,1679561615669-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:40,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37881, period=300000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:40,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-23 08:53:40,006 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-23 08:53:40,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-23 08:53:40,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-23 08:53:40,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-23 08:53:40,027 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:53:40,029 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:53:40,049 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,052 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f empty. 2023-03-23 08:53:40,053 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,053 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-23 08:53:40,077 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:40,079 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => df4c4c6f1021c2dacf2331fe9a27397f, NAME => 'hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing df4c4c6f1021c2dacf2331fe9a27397f, disabling compactions & flushes 2023-03-23 08:53:40,491 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. after waiting 0 ms 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,491 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,491 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for df4c4c6f1021c2dacf2331fe9a27397f: 2023-03-23 08:53:40,496 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:53:40,511 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679561620499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561620499"}]},"ts":"1679561620499"} 2023-03-23 08:53:40,536 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:53:40,538 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:53:40,542 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561620538"}]},"ts":"1679561620538"} 2023-03-23 08:53:40,547 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-23 08:53:40,557 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=df4c4c6f1021c2dacf2331fe9a27397f, ASSIGN}] 2023-03-23 08:53:40,560 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=df4c4c6f1021c2dacf2331fe9a27397f, ASSIGN 2023-03-23 08:53:40,561 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=df4c4c6f1021c2dacf2331fe9a27397f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:53:40,712 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=df4c4c6f1021c2dacf2331fe9a27397f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:40,713 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679561620712"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561620712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561620712"}]},"ts":"1679561620712"} 2023-03-23 08:53:40,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure df4c4c6f1021c2dacf2331fe9a27397f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:40,877 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,877 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => df4c4c6f1021c2dacf2331fe9a27397f, NAME => 'hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:40,878 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,878 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:40,878 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,878 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,881 INFO [StoreOpener-df4c4c6f1021c2dacf2331fe9a27397f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,883 DEBUG [StoreOpener-df4c4c6f1021c2dacf2331fe9a27397f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/info 2023-03-23 08:53:40,883 DEBUG [StoreOpener-df4c4c6f1021c2dacf2331fe9a27397f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/info 2023-03-23 08:53:40,883 INFO [StoreOpener-df4c4c6f1021c2dacf2331fe9a27397f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df4c4c6f1021c2dacf2331fe9a27397f columnFamilyName info 2023-03-23 08:53:40,884 INFO [StoreOpener-df4c4c6f1021c2dacf2331fe9a27397f-1] regionserver.HStore(310): Store=df4c4c6f1021c2dacf2331fe9a27397f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:40,886 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,886 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,890 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:53:40,893 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:53:40,895 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened df4c4c6f1021c2dacf2331fe9a27397f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70292496, jitterRate=0.0474398136138916}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:53:40,895 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for df4c4c6f1021c2dacf2331fe9a27397f: 2023-03-23 08:53:40,897 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f., pid=6, masterSystemTime=1679561620870 2023-03-23 08:53:40,900 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,900 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:53:40,902 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=df4c4c6f1021c2dacf2331fe9a27397f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:40,902 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679561620901"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561620901"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561620901"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561620901"}]},"ts":"1679561620901"} 2023-03-23 08:53:40,909 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-23 08:53:40,909 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure df4c4c6f1021c2dacf2331fe9a27397f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 189 msec 2023-03-23 08:53:40,912 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-23 08:53:40,913 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=df4c4c6f1021c2dacf2331fe9a27397f, ASSIGN in 352 msec 2023-03-23 08:53:40,914 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:53:40,914 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561620914"}]},"ts":"1679561620914"} 2023-03-23 08:53:40,917 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-23 08:53:40,921 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:53:40,923 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 905 msec 2023-03-23 08:53:40,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-23 08:53:40,928 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-23 08:53:40,928 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:53:40,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-23 08:53:40,980 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-23 08:53:40,986 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-03-23 08:53:41,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-23 08:53:41,011 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-23 08:53:41,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-03-23 08:53:41,028 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-23 08:53:41,030 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-23 08:53:41,031 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.016sec 2023-03-23 08:53:41,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-23 08:53:41,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-23 08:53:41,034 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-23 08:53:41,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37881,1679561615669-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-23 08:53:41,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37881,1679561615669-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-23 08:53:41,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-23 08:53:41,088 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(139): Connect 0x3c1e7f16 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:41,092 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@672b468, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:41,107 DEBUG [hconnection-0x86f282a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:41,119 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36178, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:41,130 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:53:41,131 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(139): Connect 0x64b944ad to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:41,143 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33fc1e18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:41,143 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:53:41,166 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=209, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=343, ProcessCount=169, AvailableMemoryMB=6664 2023-03-23 08:53:41,176 DEBUG [Listener at localhost/38371] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:53:41,180 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47698, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:53:41,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:53:41,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:41,196 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:53:41,198 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:53:41,201 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:41,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-23 08:53:41,202 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 empty. 2023-03-23 08:53:41,204 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:41,204 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-23 08:53:41,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:41,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:41,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:41,620 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:41,622 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1735e7d5d07578998f62eda41d807aa2, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:53:41,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:42,035 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:42,036 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 1735e7d5d07578998f62eda41d807aa2, disabling compactions & flushes 2023-03-23 08:53:42,036 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,036 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,036 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. after waiting 0 ms 2023-03-23 08:53:42,036 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,036 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,036 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:42,041 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:53:42,042 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561622042"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561622042"}]},"ts":"1679561622042"} 2023-03-23 08:53:42,045 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:53:42,046 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:53:42,047 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561622046"}]},"ts":"1679561622046"} 2023-03-23 08:53:42,055 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-23 08:53:42,060 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, ASSIGN}] 2023-03-23 08:53:42,063 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, ASSIGN 2023-03-23 08:53:42,064 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:53:42,216 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:42,217 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561622216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561622216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561622216"}]},"ts":"1679561622216"} 2023-03-23 08:53:42,221 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:42,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:42,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1735e7d5d07578998f62eda41d807aa2, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:42,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:42,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,397 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,401 DEBUG [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:42,401 DEBUG [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:42,406 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1735e7d5d07578998f62eda41d807aa2 columnFamilyName family1 2023-03-23 08:53:42,411 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] regionserver.HStore(310): Store=1735e7d5d07578998f62eda41d807aa2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:42,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:42,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:53:42,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1735e7d5d07578998f62eda41d807aa2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60475642, jitterRate=-0.09884271025657654}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:53:42,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:42,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., pid=11, masterSystemTime=1679561622381 2023-03-23 08:53:42,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:42,455 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:42,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561622455"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561622455"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561622455"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561622455"}]},"ts":"1679561622455"} 2023-03-23 08:53:42,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-23 08:53:42,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941 in 237 msec 2023-03-23 08:53:42,466 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-23 08:53:42,466 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, ASSIGN in 402 msec 2023-03-23 08:53:42,467 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:53:42,467 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561622467"}]},"ts":"1679561622467"} 2023-03-23 08:53:42,480 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-23 08:53:42,483 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:53:42,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2920 sec 2023-03-23 08:53:43,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-23 08:53:43,331 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-23 08:53:43,353 DEBUG [hconnection-0x130f47b7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:43,356 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36190, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:43,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:43,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,560 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:43,560 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:43,561 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:43,561 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:43,562 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,562 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,564 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:43,564 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:43,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:43,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,568 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:43,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:43,571 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:43,572 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:43,573 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:43,575 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:43,576 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:53:43,846 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130029c917637e5441d9406b6d90eba624e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130029c917637e5441d9406b6d90eba624e 2023-03-23 08:53:44,270 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/f6f254e6c1a34d7ba52dfe707e04f39f 2023-03-23 08:53:44,285 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/f6f254e6c1a34d7ba52dfe707e04f39f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f 2023-03-23 08:53:44,299 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-23 08:53:44,304 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 728ms, sequenceid=5, compaction requested=false 2023-03-23 08:53:44,306 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:44,310 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:44,310 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:44,310 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:44,311 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:44,311 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,311 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:44,311 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:44,315 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:44,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:44,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:44,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:44,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:44,319 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ee19d9b[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:44,319 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:44,321 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,321 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:44,325 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,325 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:44,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:44,325 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:44,325 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:44,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:44,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:44,330 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:44,330 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:44,330 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:44,330 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,330 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:44,333 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,333 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,333 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:44,334 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:44,337 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:44,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,351 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,352 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:44,352 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:44,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:44,352 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:44,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:44,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:44,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:44,357 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:44,357 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:44,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:44,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:44,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:44,457 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:44,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:44,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:44,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:44,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,503 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:44,503 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:44,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,507 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:44,507 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:44,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:44,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:44,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:44,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:44,512 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:44,512 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:44,512 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:44,512 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:53:44,583 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-23 08:53:44,587 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-23 08:53:44,980 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287296891cb56e449ca11ec2662a462bb5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511287296891cb56e449ca11ec2662a462bb5 2023-03-23 08:53:45,019 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/d9b1851c5ffe4c929834f1926c9bb167 2023-03-23 08:53:45,037 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/d9b1851c5ffe4c929834f1926c9bb167 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167 2023-03-23 08:53:45,050 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:53:45,053 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 541ms, sequenceid=9, compaction requested=false 2023-03-23 08:53:45,053 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:45,053 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:45,054 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:45,054 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:45,054 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,054 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:45,054 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:45,058 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,058 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:45,058 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,058 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:45,060 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:45,061 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@308373c2[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:45,061 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:45,061 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,062 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,063 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:45,063 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:45,063 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:45,064 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,064 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:45,066 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:45,067 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:45,067 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:45,067 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:45,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:45,077 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:45,078 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:45,078 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:45,078 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,078 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:45,083 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,083 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:45,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,083 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:45,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,086 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:45,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,090 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:45,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,099 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:45,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:45,099 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,100 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,100 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,100 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:45,100 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:45,100 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:45,201 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:45,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:45,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:45,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:45,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,224 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:45,224 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:45,225 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:45,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:45,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,228 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:45,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:45,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,229 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,229 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:45,230 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:45,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:45,232 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:45,232 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:45,232 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:45,232 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:45,358 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:53:45,670 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201f04297ee6e1d464c82cea8442c9bd3a7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f04297ee6e1d464c82cea8442c9bd3a7 2023-03-23 08:53:45,845 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-23 08:53:45,845 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-23 08:53:46,091 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/190a7202995c419898e5ae48eee6cffd 2023-03-23 08:53:46,102 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/190a7202995c419898e5ae48eee6cffd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd 2023-03-23 08:53:46,110 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd, entries=1, sequenceid=13, filesize=5.0 K 2023-03-23 08:53:46,112 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 880ms, sequenceid=13, compaction requested=true 2023-03-23 08:53:46,112 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:46,112 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:46,112 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,117 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:46,118 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:46,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a1c5c1[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:46,121 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:46,121 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,123 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,123 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:46,123 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:46,123 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:46,123 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,123 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:46,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:46,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:46,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:46,130 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:46,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:46,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:46,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,130 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:46,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:46,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,141 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,143 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,149 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:46,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:46,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:46,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,149 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:46,150 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:46,151 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:46,150 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:46,151 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:46,150 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:46,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:46,251 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:46,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:46,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:46,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:46,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,272 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:46,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:46,273 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:46,273 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:46,274 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,274 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,276 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:46,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:46,276 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:46,276 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,276 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:46,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:46,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:46,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,278 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:46,278 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:46,278 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:46,278 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:46,278 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:46,278 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:46,278 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:46,279 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:46,733 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051dadc3ef882f43159e15eeb531fc8299 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512051dadc3ef882f43159e15eeb531fc8299 2023-03-23 08:53:46,759 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/05a2ea4ddd7c4c8796aadf824c729761 2023-03-23 08:53:46,774 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/05a2ea4ddd7c4c8796aadf824c729761 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761 2023-03-23 08:53:46,784 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761, entries=1, sequenceid=17, filesize=5.0 K 2023-03-23 08:53:46,786 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 507ms, sequenceid=17, compaction requested=true 2023-03-23 08:53:46,786 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:46,786 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:46,786 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:46,787 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,788 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:46,788 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:46,791 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,791 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,792 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:46,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:46,795 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34cdb667[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:46,795 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:46,795 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,806 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:46,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,807 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:46,807 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:46,807 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:46,809 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,809 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:46,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,809 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:46,810 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:46,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:46,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:46,819 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:46,819 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:46,819 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:46,819 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,819 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:46,822 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:46,822 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,823 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:46,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:46,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:46,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:46,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:46,824 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:46,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,827 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,841 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:46,841 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:46,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:46,841 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:46,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:46,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:46,841 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:46,841 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:46,842 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:46,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:46,943 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:46,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:46,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:46,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:46,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,963 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:46,963 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:46,964 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:46,964 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:46,965 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,965 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,966 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:46,966 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,967 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:46,967 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:46,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:46,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:46,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:46,970 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:46,970 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:46,970 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:46,974 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:46,974 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:46,974 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:46,974 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:46,975 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:47,414 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ac2cb6043aa74b788f341a0d99444d94 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac2cb6043aa74b788f341a0d99444d94 2023-03-23 08:53:47,456 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/52ec0653427b408ebfa4380ca092f0b0 2023-03-23 08:53:47,467 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/52ec0653427b408ebfa4380ca092f0b0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0 2023-03-23 08:53:47,480 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0, entries=1, sequenceid=21, filesize=5.0 K 2023-03-23 08:53:47,482 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 507ms, sequenceid=21, compaction requested=true 2023-03-23 08:53:47,482 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:47,482 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:47,482 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:47,483 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:47,483 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,483 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:47,483 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:47,488 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,488 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:47,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:47,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16e2e820[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:47,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:47,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,497 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,497 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:47,497 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:47,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:47,497 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:47,500 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:47,500 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:47,500 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:47,501 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:47,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:47,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:47,505 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:47,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:47,506 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:47,506 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,506 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:47,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:47,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:47,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:47,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,512 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:47,516 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:47,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:47,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:47,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:47,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:47,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:47,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:47,524 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:47,524 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:47,625 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:47,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:47,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:47,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:47,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,650 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:47,650 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:47,651 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:47,651 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:47,654 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,654 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,655 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,656 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:47,656 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:47,660 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:47,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:47,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:47,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:47,712 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:47,712 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:47,715 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:47,720 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:47,720 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:47,720 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:47,720 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:47,720 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:47,757 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311654c6609f534112ac9e9b8f0c526e3d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311654c6609f534112ac9e9b8f0c526e3d 2023-03-23 08:53:48,193 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/a3af450faf5b436e8e036542054dad5d 2023-03-23 08:53:48,206 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/a3af450faf5b436e8e036542054dad5d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d 2023-03-23 08:53:48,216 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d, entries=1, sequenceid=25, filesize=5.0 K 2023-03-23 08:53:48,218 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 498ms, sequenceid=25, compaction requested=true 2023-03-23 08:53:48,218 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:48,218 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:48,218 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:48,219 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:48,222 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:48,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:48,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:48,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:48,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:48,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d54845d[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:48,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:48,227 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,228 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:48,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:48,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:48,229 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,229 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:48,230 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:48,230 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:48,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:48,231 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:48,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:48,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:48,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:48,238 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:48,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:48,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:48,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,238 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:48,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:48,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:48,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:48,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:48,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:48,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:48,241 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:48,249 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,255 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:48,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:48,255 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:48,255 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:48,256 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:48,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:48,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:48,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:48,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,257 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:48,257 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:48,358 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:48,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:48,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:48,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:48,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,373 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:48,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:48,374 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:48,374 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:48,374 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,374 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,376 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:48,376 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:48,377 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,377 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,379 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,379 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,383 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:48,383 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,383 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:48,383 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:48,384 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:48,384 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:48,384 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:48,384 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:48,384 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:48,385 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:48,834 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601015dd38cbc5af34f2d883104cc155161b4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601015dd38cbc5af34f2d883104cc155161b4 2023-03-23 08:53:48,853 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/afe5694ffa9947efb444579bee741b0e 2023-03-23 08:53:48,869 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/afe5694ffa9947efb444579bee741b0e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e 2023-03-23 08:53:48,883 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e, entries=1, sequenceid=29, filesize=5.0 K 2023-03-23 08:53:48,885 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 500ms, sequenceid=29, compaction requested=true 2023-03-23 08:53:48,886 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:48,887 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:48,887 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:48,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:48,891 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:48,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:48,891 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:48,892 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,892 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:48,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:48,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:48,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3254c1fb[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:48,895 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:48,895 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,896 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,897 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:48,897 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:48,897 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:48,899 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:48,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,899 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:48,899 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:48,901 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:48,901 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:48,901 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:48,901 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:48,901 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:48,901 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,901 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:48,906 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,906 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:48,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,907 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:48,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:48,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:48,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:48,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:48,908 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:48,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,911 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:48,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,918 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:48,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:48,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:48,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:48,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:48,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:48,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:48,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:48,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:48,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:48,920 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,920 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,920 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:48,921 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:48,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:48,921 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:49,021 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:49,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:49,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:49,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:49,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,047 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:49,047 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,047 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:49,047 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:49,048 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,048 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,050 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:49,050 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:49,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,051 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,051 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:49,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:49,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,059 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:49,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:49,064 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:49,064 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:49,064 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:49,064 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:49,064 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:49,104 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103aa26edfbf379463fb04af9ab3dc41d85 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103aa26edfbf379463fb04af9ab3dc41d85 2023-03-23 08:53:49,132 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/fcadfb953ac4482f8f26f02b89219a8f 2023-03-23 08:53:49,149 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/fcadfb953ac4482f8f26f02b89219a8f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f 2023-03-23 08:53:49,158 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f, entries=1, sequenceid=33, filesize=5.0 K 2023-03-23 08:53:49,160 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 96ms, sequenceid=33, compaction requested=true 2023-03-23 08:53:49,160 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:49,160 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:49,160 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:49,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:49,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:49,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:49,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,163 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:49,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:49,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bf32758[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:49,166 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:49,166 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,167 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,168 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:49,168 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,168 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:49,168 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:49,168 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:49,169 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,169 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:49,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,169 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:49,170 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:49,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:49,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:49,174 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:49,174 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:49,174 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:49,174 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,174 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:49,177 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,177 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:49,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,177 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:49,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:49,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,179 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,181 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,184 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:49,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,185 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:49,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:49,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:49,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:49,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:49,187 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:49,187 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:49,287 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:49,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:49,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:49,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:49,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,302 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:49,302 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,303 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:49,303 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:49,303 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,304 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,311 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,311 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:49,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:49,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:49,311 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,311 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:49,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:49,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:49,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,314 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:49,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:49,315 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:49,316 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:49,316 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:49,744 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090734ac14199bf84cd6a1755237b868f56a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090734ac14199bf84cd6a1755237b868f56a 2023-03-23 08:53:49,789 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/e867dfffef0543a8936d34a354f80597 2023-03-23 08:53:49,810 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/e867dfffef0543a8936d34a354f80597 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597 2023-03-23 08:53:49,824 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597, entries=1, sequenceid=37, filesize=5.0 K 2023-03-23 08:53:49,826 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 510ms, sequenceid=37, compaction requested=true 2023-03-23 08:53:49,826 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:49,826 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:49,826 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:49,826 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:49,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:49,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:49,830 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,830 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,830 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,830 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:49,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:49,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ad161ee[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:49,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:49,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,834 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,834 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:49,834 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:49,834 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:49,834 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,834 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:49,835 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:49,835 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:49,835 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:49,835 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:49,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:49,840 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:49,840 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:49,840 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:49,840 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,841 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:49,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:49,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:49,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:49,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,843 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:49,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,845 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:49,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,849 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:49,849 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:49,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:49,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:49,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:49,849 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:49,850 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:49,851 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:49,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:49,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:49,951 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:49,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:49,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:49,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:49,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,970 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:49,971 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,971 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-23 08:53:49,971 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-23 08:53:49,972 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,972 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,974 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:49,974 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:49,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,975 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:49,975 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:49,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:49,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,988 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-23 08:53:49,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:49,988 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-23 08:53:49,988 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:49,990 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-23 08:53:49,990 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:49,990 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:49,990 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:49,990 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. started... 2023-03-23 08:53:49,990 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1735e7d5d07578998f62eda41d807aa2 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:50,033 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120764f3f859efc4d799ee3dae941043cb9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120764f3f859efc4d799ee3dae941043cb9 2023-03-23 08:53:50,467 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/b885aa0de3de4011bfa566cfdbd8287d 2023-03-23 08:53:50,479 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/b885aa0de3de4011bfa566cfdbd8287d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d 2023-03-23 08:53:50,487 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d, entries=1, sequenceid=41, filesize=5.0 K 2023-03-23 08:53:50,488 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1735e7d5d07578998f62eda41d807aa2 in 498ms, sequenceid=41, compaction requested=true 2023-03-23 08:53:50,488 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:50,488 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-23 08:53:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-23 08:53:50,492 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,492 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:50,492 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,492 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:50,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:50,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-23 08:53:50,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@733e6685[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:50,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-23 08:53:50,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,496 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-23 08:53:50,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-23 08:53:50,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:50,500 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,500 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:50,505 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,505 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-23 08:53:50,505 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:50,505 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-23 08:53:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:50,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:50,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:50,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:50,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-23 08:53:50,509 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-23 08:53:50,509 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:50,509 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:50,509 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,509 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:50,511 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,512 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:50,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:50,512 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:50,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:50,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:50,513 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:50,518 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:50,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:50,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:50,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:50,521 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:50,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-23 08:53:50,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:50,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:50,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:50,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:50,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:50,622 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:50,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:50,630 DEBUG [Listener at localhost/38371] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-23 08:53:50,633 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-23 08:53:50,635 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:50,651 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:50,652 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:50,654 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:53:50,654 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562230654 2023-03-23 08:53:50,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:53:50,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-23 08:53:50,662 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:50,663 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x5c3492d1 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:50,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@9f6beda, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:50,715 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,376 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038856e8f54a7a445dbf19a6f481a8739e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038856e8f54a7a445dbf19a6f481a8739e 2023-03-23 08:53:51,379 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:51,413 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:51,415 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:51,442 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:53:51,445 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53124, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:53:51,519 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5e624edf64f749a8b85b87153fc61be3 first=Optional[r3] last=Optional[r6] 2023-03-23 08:53:51,543 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5e624edf64f749a8b85b87153fc61be3}] 2023-03-23 08:53:51,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5e624edf64f749a8b85b87153fc61be3 for inclusion in 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:51,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-23 08:53:51,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:53:51,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:51,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5e624edf64f749a8b85b87153fc61be3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9tg48gub7gimfq4ccmhqhhvfvu4b9f7uln8rbv8gosm096qrkica59qqfafltc8d/family1/5e624edf64f749a8b85b87153fc61be3 2023-03-23 08:53:51,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9tg48gub7gimfq4ccmhqhhvfvu4b9f7uln8rbv8gosm096qrkica59qqfafltc8d/family1/5e624edf64f749a8b85b87153fc61be3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_ 2023-03-23 08:53:51,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9tg48gub7gimfq4ccmhqhhvfvu4b9f7uln8rbv8gosm096qrkica59qqfafltc8d/family1/5e624edf64f749a8b85b87153fc61be3 into 1735e7d5d07578998f62eda41d807aa2/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_ - updating store file list. 2023-03-23 08:53:51,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_ into 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:51,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9tg48gub7gimfq4ccmhqhhvfvu4b9f7uln8rbv8gosm096qrkica59qqfafltc8d/family1/5e624edf64f749a8b85b87153fc61be3 into 1735e7d5d07578998f62eda41d807aa2/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_) 2023-03-23 08:53:51,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__9tg48gub7gimfq4ccmhqhhvfvu4b9f7uln8rbv8gosm096qrkica59qqfafltc8d/family1/5e624edf64f749a8b85b87153fc61be3 2023-03-23 08:53:51,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:53:51,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:53:51,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 1735e7d5d07578998f62eda41d807aa2/family1 is initiating minor compaction 2023-03-23 08:53:51,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4754f175; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:51,634 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1735e7d5d07578998f62eda41d807aa2/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:51,635 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp, totalSize=50.2 K 2023-03-23 08:53:51,637 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting d9b1851c5ffe4c929834f1926c9bb167, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:53:51,637 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 190a7202995c419898e5ae48eee6cffd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:53:51,637 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 05a2ea4ddd7c4c8796aadf824c729761, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:53:51,637 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 52ec0653427b408ebfa4380ca092f0b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting a3af450faf5b436e8e036542054dad5d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting afe5694ffa9947efb444579bee741b0e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting fcadfb953ac4482f8f26f02b89219a8f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting e867dfffef0543a8936d34a354f80597, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting b885aa0de3de4011bfa566cfdbd8287d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:53:51,638 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:53:51,652 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:51,655 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:51,663 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac2cb6043aa74b788f341a0d99444d94 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac2cb6043aa74b788f341a0d99444d94 2023-03-23 08:53:51,666 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311654c6609f534112ac9e9b8f0c526e3d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311654c6609f534112ac9e9b8f0c526e3d 2023-03-23 08:53:51,669 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601015dd38cbc5af34f2d883104cc155161b4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601015dd38cbc5af34f2d883104cc155161b4 2023-03-23 08:53:51,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103aa26edfbf379463fb04af9ab3dc41d85 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103aa26edfbf379463fb04af9ab3dc41d85 2023-03-23 08:53:51,676 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-23 08:53:51,676 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,677 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:51,726 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1735e7d5d07578998f62eda41d807aa2#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:53:52,165 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/f41d9026e8234c58890ff280dc1db9e5 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f41d9026e8234c58890ff280dc1db9e5 2023-03-23 08:53:52,186 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1735e7d5d07578998f62eda41d807aa2/family1 of 1735e7d5d07578998f62eda41d807aa2 into f41d9026e8234c58890ff280dc1db9e5(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:53:52,186 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:52,189 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1679561631625; duration=0sec 2023-03-23 08:53:52,191 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:52,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_] to archive 2023-03-23 08:53:52,241 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050b18fa2bf3b340a1aab1ce3cede1fb32 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512050b18fa2bf3b340a1aab1ce3cede1fb32 2023-03-23 08:53:52,242 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:52,244 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:52,247 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/d9b1851c5ffe4c929834f1926c9bb167 2023-03-23 08:53:52,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/190a7202995c419898e5ae48eee6cffd 2023-03-23 08:53:52,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/05a2ea4ddd7c4c8796aadf824c729761 2023-03-23 08:53:52,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/52ec0653427b408ebfa4380ca092f0b0 2023-03-23 08:53:52,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/a3af450faf5b436e8e036542054dad5d 2023-03-23 08:53:52,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/afe5694ffa9947efb444579bee741b0e 2023-03-23 08:53:52,260 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/fcadfb953ac4482f8f26f02b89219a8f 2023-03-23 08:53:52,262 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/e867dfffef0543a8936d34a354f80597 2023-03-23 08:53:52,265 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/b885aa0de3de4011bfa566cfdbd8287d 2023-03-23 08:53:52,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/7e31663facd942d7ae9a83c3813c4a0f_SeqId_44_ 2023-03-23 08:53:52,312 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb946d6eb8a840c2abb7a9d07c4665fd first=Optional[r00] last=Optional[r2] 2023-03-23 08:53:52,316 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb946d6eb8a840c2abb7a9d07c4665fd}] 2023-03-23 08:53:52,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb946d6eb8a840c2abb7a9d07c4665fd for inclusion in 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:52,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-23 08:53:52,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:53:52,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:52,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/bb946d6eb8a840c2abb7a9d07c4665fd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__50q1e6olpa3c5fnqscbp3bno3u1genpp6hg09od6bt5h4pgkpai0b8b18fsrmgqt/family1/bb946d6eb8a840c2abb7a9d07c4665fd 2023-03-23 08:53:52,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__50q1e6olpa3c5fnqscbp3bno3u1genpp6hg09od6bt5h4pgkpai0b8b18fsrmgqt/family1/bb946d6eb8a840c2abb7a9d07c4665fd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_ 2023-03-23 08:53:52,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__50q1e6olpa3c5fnqscbp3bno3u1genpp6hg09od6bt5h4pgkpai0b8b18fsrmgqt/family1/bb946d6eb8a840c2abb7a9d07c4665fd into 1735e7d5d07578998f62eda41d807aa2/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_ - updating store file list. 2023-03-23 08:53:52,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_ into 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:52,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__50q1e6olpa3c5fnqscbp3bno3u1genpp6hg09od6bt5h4pgkpai0b8b18fsrmgqt/family1/bb946d6eb8a840c2abb7a9d07c4665fd into 1735e7d5d07578998f62eda41d807aa2/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_) 2023-03-23 08:53:52,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__50q1e6olpa3c5fnqscbp3bno3u1genpp6hg09od6bt5h4pgkpai0b8b18fsrmgqt/family1/bb946d6eb8a840c2abb7a9d07c4665fd 2023-03-23 08:53:52,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:53:52,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:53:52,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 1735e7d5d07578998f62eda41d807aa2/family1 is initiating minor compaction (all files) 2023-03-23 08:53:52,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3a1e6e36; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:52,347 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1735e7d5d07578998f62eda41d807aa2/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:52,348 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f41d9026e8234c58890ff280dc1db9e5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp, totalSize=358.6 K 2023-03-23 08:53:52,349 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f6f254e6c1a34d7ba52dfe707e04f39f, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:53:52,349 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f41d9026e8234c58890ff280dc1db9e5, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:53:52,350 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting ca4292f959cd466498e01871035e099c_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841145 2023-03-23 08:53:52,352 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:52,357 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:52,360 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130029c917637e5441d9406b6d90eba624e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130029c917637e5441d9406b6d90eba624e 2023-03-23 08:53:52,363 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f04297ee6e1d464c82cea8442c9bd3a7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201f04297ee6e1d464c82cea8442c9bd3a7 2023-03-23 08:53:52,366 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512051dadc3ef882f43159e15eeb531fc8299 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512051dadc3ef882f43159e15eeb531fc8299 2023-03-23 08:53:52,366 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-23 08:53:52,366 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-23 08:53:52,367 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:52,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:52,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:53:52,371 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:52,375 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.7300 sec 2023-03-23 08:53:52,418 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1735e7d5d07578998f62eda41d807aa2#family1#compaction#11 average throughput is 3.84 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:53:52,513 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/.tmp/family1/cec886294cb9445aa148fb6698fc60b0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/cec886294cb9445aa148fb6698fc60b0 2023-03-23 08:53:52,529 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1735e7d5d07578998f62eda41d807aa2/family1 of 1735e7d5d07578998f62eda41d807aa2 into cec886294cb9445aa148fb6698fc60b0(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:53:52,530 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:52,530 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1679561632347; duration=0sec 2023-03-23 08:53:52,530 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:52,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f41d9026e8234c58890ff280dc1db9e5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_] to archive 2023-03-23 08:53:52,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:52,635 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f6f254e6c1a34d7ba52dfe707e04f39f 2023-03-23 08:53:52,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f41d9026e8234c58890ff280dc1db9e5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/f41d9026e8234c58890ff280dc1db9e5 2023-03-23 08:53:52,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/ca4292f959cd466498e01871035e099c_SeqId_47_ 2023-03-23 08:53:52,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-23 08:53:52,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:52,706 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-23 08:53:52,709 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:52,709 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-23 08:53:52,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-23 08:53:52,733 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, REOPEN/MOVE}] 2023-03-23 08:53:52,735 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, REOPEN/MOVE 2023-03-23 08:53:52,736 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:52,736 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561632736"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561632736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561632736"}]},"ts":"1679561632736"} 2023-03-23 08:53:52,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:52,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:52,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1735e7d5d07578998f62eda41d807aa2, disabling compactions & flushes 2023-03-23 08:53:52,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:52,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:52,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. after waiting 0 ms 2023-03-23 08:53:52,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:52,913 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-23 08:53:52,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:52,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:52,915 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1735e7d5d07578998f62eda41d807aa2 to self. 2023-03-23 08:53:52,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:52,921 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=CLOSED 2023-03-23 08:53:52,922 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561632921"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561632921"}]},"ts":"1679561632921"} 2023-03-23 08:53:52,932 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-23 08:53:52,933 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941 in 185 msec 2023-03-23 08:53:52,937 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=true 2023-03-23 08:53:53,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:53,089 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561633088"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561633088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561633088"}]},"ts":"1679561633088"} 2023-03-23 08:53:53,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:53,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:53,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1735e7d5d07578998f62eda41d807aa2, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:53,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:53,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,251 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,254 DEBUG [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:53,254 DEBUG [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:53,254 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1735e7d5d07578998f62eda41d807aa2 columnFamilyName family1 2023-03-23 08:53:53,295 DEBUG [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/cec886294cb9445aa148fb6698fc60b0 2023-03-23 08:53:53,296 INFO [StoreOpener-1735e7d5d07578998f62eda41d807aa2-1] regionserver.HStore(310): Store=1735e7d5d07578998f62eda41d807aa2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:53,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:53:53,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1735e7d5d07578998f62eda41d807aa2; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63571607, jitterRate=-0.05270923674106598}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:53:53,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:53,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., pid=17, masterSystemTime=1679561633245 2023-03-23 08:53:53,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:53,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:53:53,316 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=1735e7d5d07578998f62eda41d807aa2, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:53,317 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679561633316"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561633316"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561633316"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561633316"}]},"ts":"1679561633316"} 2023-03-23 08:53:53,321 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-23 08:53:53,322 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 1735e7d5d07578998f62eda41d807aa2, server=jenkins-hbase4.apache.org,32923,1679561616941 in 227 msec 2023-03-23 08:53:53,336 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-23 08:53:53,336 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=1735e7d5d07578998f62eda41d807aa2, REOPEN/MOVE in 589 msec 2023-03-23 08:53:53,341 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-23 08:53:53,342 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 624 msec 2023-03-23 08:53:53,346 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 678 msec 2023-03-23 08:53:53,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-23 08:53:53,361 INFO [Listener at localhost/38371] compactions.TestMobCompactor(278): alter status finished 2023-03-23 08:53:53,367 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-23 08:53:53,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:53,372 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:53,373 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:53:53,373 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562233373 2023-03-23 08:53:53,375 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:53:53,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-23 08:53:53,376 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:53,376 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x78ff69ed to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:53,414 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ecb2a15, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:53,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,415 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,594 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289c50145dac0b4dee9975d685dbc0b267 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289c50145dac0b4dee9975d685dbc0b267 2023-03-23 08:53:53,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:53,619 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:53,621 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36172, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:53,638 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:53:53,645 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34362, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:53:53,687 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a0a9cebff8744dce83c0b4947a82b063 first=Optional[r05] last=Optional[r8] 2023-03-23 08:53:53,689 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=51 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a0a9cebff8744dce83c0b4947a82b063}] 2023-03-23 08:53:53,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a0a9cebff8744dce83c0b4947a82b063 for inclusion in 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:53,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-23 08:53:53,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:53:53,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:53:53,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/a0a9cebff8744dce83c0b4947a82b063 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__qco92ng816li935b87tlbpkls30bsd0vnmkmm5f8kid33fovmc47ul6th8eg6kpk/family1/a0a9cebff8744dce83c0b4947a82b063 2023-03-23 08:53:53,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__qco92ng816li935b87tlbpkls30bsd0vnmkmm5f8kid33fovmc47ul6th8eg6kpk/family1/a0a9cebff8744dce83c0b4947a82b063 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/0b1aeca06eec45728e1e22d13205f965_SeqId_53_ 2023-03-23 08:53:53,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__qco92ng816li935b87tlbpkls30bsd0vnmkmm5f8kid33fovmc47ul6th8eg6kpk/family1/a0a9cebff8744dce83c0b4947a82b063 into 1735e7d5d07578998f62eda41d807aa2/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/0b1aeca06eec45728e1e22d13205f965_SeqId_53_ - updating store file list. 2023-03-23 08:53:53,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/0b1aeca06eec45728e1e22d13205f965_SeqId_53_ into 1735e7d5d07578998f62eda41d807aa2/family1 2023-03-23 08:53:53,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__qco92ng816li935b87tlbpkls30bsd0vnmkmm5f8kid33fovmc47ul6th8eg6kpk/family1/a0a9cebff8744dce83c0b4947a82b063 into 1735e7d5d07578998f62eda41d807aa2/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/family1/0b1aeca06eec45728e1e22d13205f965_SeqId_53_) 2023-03-23 08:53:53,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__qco92ng816li935b87tlbpkls30bsd0vnmkmm5f8kid33fovmc47ul6th8eg6kpk/family1/a0a9cebff8744dce83c0b4947a82b063 2023-03-23 08:53:53,736 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:53,737 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:53,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120764f3f859efc4d799ee3dae941043cb9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120764f3f859efc4d799ee3dae941043cb9 2023-03-23 08:53:53,745 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511287296891cb56e449ca11ec2662a462bb5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511287296891cb56e449ca11ec2662a462bb5 2023-03-23 08:53:53,748 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-23 08:53:53,748 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-23 08:53:53,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,749 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-23 08:53:53,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-23 08:53:53,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:53:53,751 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-23 08:53:53,753 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 382 msec 2023-03-23 08:53:53,848 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=249 (was 209) Potentially hanging thread: (jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x62d0adf7-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x322a1d1a-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1370277958_15 at /127.0.0.1:50154 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x62d0adf7-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x62d0adf7-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50052 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50222 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50320 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50250 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x62d0adf7-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x78ff69ed-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x5c3492d1-SendThread(127.0.0.1:50031) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50280 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50126 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x5c3492d1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS:0;jenkins-hbase4:32923-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x78ff69ed sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/219019176.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x322a1d1a-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50138 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50276 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50174 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50168 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50214 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50080 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x322a1d1a-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x78ff69ed-SendThread(127.0.0.1:50031) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x62d0adf7-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x5c3492d1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/219019176.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50106 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50136 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50268 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=596 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=404 (was 343) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=6242 (was 6664) 2023-03-23 08:53:53,870 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=249, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=404, ProcessCount=169, AvailableMemoryMB=6239 2023-03-23 08:53:53,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-23 08:53:53,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:53,876 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:53:53,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-23 08:53:53,877 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:53:53,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-23 08:53:53,880 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:53,881 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe empty. 2023-03-23 08:53:53,882 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:53,882 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-23 08:53:53,903 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-23 08:53:53,905 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => bdaddcf1cd39563f63add7141876dbbe, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing bdaddcf1cd39563f63add7141876dbbe, disabling compactions & flushes 2023-03-23 08:53:53,929 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. after waiting 0 ms 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:53,929 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:53,929 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:53,933 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:53:53,934 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561633934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561633934"}]},"ts":"1679561633934"} 2023-03-23 08:53:53,937 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:53:53,938 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:53:53,938 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561633938"}]},"ts":"1679561633938"} 2023-03-23 08:53:53,941 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-23 08:53:53,944 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bdaddcf1cd39563f63add7141876dbbe, ASSIGN}] 2023-03-23 08:53:53,947 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bdaddcf1cd39563f63add7141876dbbe, ASSIGN 2023-03-23 08:53:53,948 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bdaddcf1cd39563f63add7141876dbbe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:53:53,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-23 08:53:54,099 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=bdaddcf1cd39563f63add7141876dbbe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,100 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561634099"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561634099"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561634099"}]},"ts":"1679561634099"} 2023-03-23 08:53:54,102 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure bdaddcf1cd39563f63add7141876dbbe, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:53:54,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-23 08:53:54,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bdaddcf1cd39563f63add7141876dbbe, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:53:54,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:53:54,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,261 INFO [StoreOpener-bdaddcf1cd39563f63add7141876dbbe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,263 DEBUG [StoreOpener-bdaddcf1cd39563f63add7141876dbbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:53:54,263 DEBUG [StoreOpener-bdaddcf1cd39563f63add7141876dbbe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:53:54,263 INFO [StoreOpener-bdaddcf1cd39563f63add7141876dbbe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bdaddcf1cd39563f63add7141876dbbe columnFamilyName family1 2023-03-23 08:53:54,265 INFO [StoreOpener-bdaddcf1cd39563f63add7141876dbbe-1] regionserver.HStore(310): Store=bdaddcf1cd39563f63add7141876dbbe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:53:54,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:53:54,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:53:54,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bdaddcf1cd39563f63add7141876dbbe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71693389, jitterRate=0.06831474602222443}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:53:54,285 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:54,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., pid=21, masterSystemTime=1679561634255 2023-03-23 08:53:54,288 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,289 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=bdaddcf1cd39563f63add7141876dbbe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,289 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561634289"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561634289"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561634289"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561634289"}]},"ts":"1679561634289"} 2023-03-23 08:53:54,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-23 08:53:54,295 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure bdaddcf1cd39563f63add7141876dbbe, server=jenkins-hbase4.apache.org,32923,1679561616941 in 190 msec 2023-03-23 08:53:54,297 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-23 08:53:54,297 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=bdaddcf1cd39563f63add7141876dbbe, ASSIGN in 351 msec 2023-03-23 08:53:54,298 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:53:54,298 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561634298"}]},"ts":"1679561634298"} 2023-03-23 08:53:54,300 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-23 08:53:54,303 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:53:54,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 430 msec 2023-03-23 08:53:54,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-23 08:53:54,482 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-23 08:53:54,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:54,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,536 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:54,536 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:54,537 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:54,537 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:54,537 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,537 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,539 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,539 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:54,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:54,540 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,540 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:54,541 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:54,542 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:54,542 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:54,542 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:54,542 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,542 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:54,543 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:53:54,633 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130eb1a569302db4b748892bd709013e054 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb1a569302db4b748892bd709013e054 2023-03-23 08:53:54,638 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/80299271449940e6916dbe9a24f70463 2023-03-23 08:53:54,648 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/80299271449940e6916dbe9a24f70463 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463 2023-03-23 08:53:54,656 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-23 08:53:54,658 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 114ms, sequenceid=5, compaction requested=false 2023-03-23 08:53:54,658 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:54,658 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:54,658 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,658 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:54,661 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,661 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:54,662 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,662 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:54,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:54,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:54,663 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:54,663 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67712a00[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:54,663 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:54,663 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,665 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:54,665 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,665 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:54,668 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:54,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:54,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:54,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:54,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:54,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:54,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:54,672 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:54,672 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:54,672 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:54,673 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,673 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:54,674 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,674 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,675 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:54,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:54,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:54,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,676 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,679 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:54,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:54,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:54,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:54,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:54,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:54,684 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:53:54,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:54,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:54,684 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:54,684 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:54,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,685 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,685 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:54,685 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,685 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,686 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,686 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,785 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:54,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:54,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:54,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:54,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,803 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:54,803 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:54,804 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:54,804 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:54,804 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,804 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:54,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:54,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:54,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:54,808 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:54,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:54,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:54,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:54,825 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:54,825 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:54,826 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:54,826 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:54,826 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:54,827 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:54,827 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:54,827 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:53:55,269 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128bd754fd829d74db295a350306923ea12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bd754fd829d74db295a350306923ea12 2023-03-23 08:53:55,277 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/76a1965d3c234776b9127748274b9fb8 2023-03-23 08:53:55,284 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/76a1965d3c234776b9127748274b9fb8 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8 2023-03-23 08:53:55,290 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:53:55,291 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 464ms, sequenceid=9, compaction requested=false 2023-03-23 08:53:55,292 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:55,292 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:55,292 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:55,292 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:55,292 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,292 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:55,292 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,294 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:55,302 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@154a0851[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:55,302 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:55,302 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,303 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:55,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:55,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:55,304 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,304 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:55,305 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:55,305 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,305 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:55,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,305 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:55,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:55,309 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:55,309 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:55,309 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:55,309 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:55,309 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,309 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:55,312 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,312 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,312 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,313 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,315 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:55,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,321 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:55,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:55,321 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:53:55,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:55,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:55,321 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:55,322 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:55,323 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:55,423 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:55,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:55,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:55,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:55,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,433 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:55,433 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:55,433 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:55,433 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:55,434 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,434 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,435 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:55,435 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:55,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:55,436 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,436 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:55,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:55,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,437 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:55,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,437 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:55,437 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:55,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:55,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:55,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:55,439 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:55,439 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:55,439 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:55,454 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e21bbb120d114290b7cb93cb0d4449df to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21bbb120d114290b7cb93cb0d4449df 2023-03-23 08:53:55,861 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/1be50041ff0c450184dd6d0ff5a133c9 2023-03-23 08:53:55,869 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/1be50041ff0c450184dd6d0ff5a133c9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9 2023-03-23 08:53:55,875 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9, entries=1, sequenceid=13, filesize=4.9 K 2023-03-23 08:53:55,876 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 437ms, sequenceid=13, compaction requested=true 2023-03-23 08:53:55,876 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:55,876 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:55,876 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:55,877 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:55,877 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,877 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:55,877 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:55,879 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,880 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,880 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:55,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:55,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74f5d507[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:55,881 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:55,882 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,883 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,883 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:55,883 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:55,883 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:55,883 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,883 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:55,885 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:55,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:55,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:55,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:55,888 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:55,888 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:55,888 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:55,888 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,888 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:55,890 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,890 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:55,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,890 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:55,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:55,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:55,890 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:55,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:55,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,900 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:55,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:55,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:55,904 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:55,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:55,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:55,904 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:55,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:55,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:55,906 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:55,906 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:56,006 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:56,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:56,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:56,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:56,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,015 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:56,015 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,016 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:56,016 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:56,016 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,016 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,018 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:56,018 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:56,019 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,019 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:56,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,019 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,020 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:56,020 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,020 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:56,020 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,020 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:56,021 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:56,021 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:56,021 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:56,021 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:56,021 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:56,447 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058944ef4e629c468e85827ab518142fab to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512058944ef4e629c468e85827ab518142fab 2023-03-23 08:53:56,466 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/bb9e74174fad417ab22c96a77a4347bf 2023-03-23 08:53:56,475 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/bb9e74174fad417ab22c96a77a4347bf as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf 2023-03-23 08:53:56,487 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf, entries=1, sequenceid=17, filesize=4.9 K 2023-03-23 08:53:56,488 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 467ms, sequenceid=17, compaction requested=true 2023-03-23 08:53:56,489 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:56,489 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:56,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:56,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:56,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:56,489 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:56,491 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,491 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,493 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,493 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@747f8f52[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:56,495 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:56,495 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,498 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:56,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:56,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:56,498 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,498 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:56,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:56,501 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:56,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:56,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:56,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:56,505 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:56,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:56,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:56,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,505 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:56,511 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,511 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,511 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:56,512 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,513 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:56,522 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:56,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:56,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:56,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:56,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:56,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:56,524 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:56,524 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:56,523 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,524 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,524 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,524 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:56,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:56,624 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:56,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:56,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:56,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:56,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,638 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:56,638 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,638 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:56,638 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:56,639 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,639 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,641 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,641 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:56,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:56,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:56,641 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,641 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:56,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:56,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:56,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,643 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:56,643 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,643 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:56,644 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:56,644 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:56,644 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:56,644 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:56,644 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:56,664 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f36294972265400298843afd9206c5e9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228f36294972265400298843afd9206c5e9 2023-03-23 08:53:56,671 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/bdfddc9d82df46438fce2b22d67379e4 2023-03-23 08:53:56,677 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/bdfddc9d82df46438fce2b22d67379e4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4 2023-03-23 08:53:56,683 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4, entries=1, sequenceid=21, filesize=4.9 K 2023-03-23 08:53:56,684 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 40ms, sequenceid=21, compaction requested=true 2023-03-23 08:53:56,684 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:56,684 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:56,685 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:56,685 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:56,685 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,685 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:56,685 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:56,686 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,686 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,687 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:56,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:56,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ab08af7[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:56,688 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:56,689 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,690 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,690 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:56,690 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:56,690 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:56,690 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,690 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:56,693 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:56,693 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:56,693 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:56,694 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:56,697 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:56,698 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:56,698 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:56,698 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:56,698 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,698 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:56,714 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,714 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,715 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:56,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:56,715 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:56,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:56,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:56,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,717 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:56,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,721 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:56,721 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:56,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:56,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:56,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:56,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:56,721 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:56,721 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:56,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:56,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:56,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:53:56,722 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,722 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,722 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,722 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,722 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,723 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,723 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,822 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:56,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:56,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:56,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:56,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,831 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:56,831 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:56,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:56,833 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,833 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,834 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,834 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:56,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:56,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:56,834 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:56,834 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:56,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,836 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,836 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:56,836 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:56,836 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:56,836 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:56,837 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:56,837 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:56,837 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:56,837 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:56,837 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:56,838 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:57,256 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231eb5157a502d54bc6b7197d600cfdd8ad to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231eb5157a502d54bc6b7197d600cfdd8ad 2023-03-23 08:53:57,282 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/dbb8804f01074e358682f9b8c407f6cd 2023-03-23 08:53:57,294 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/dbb8804f01074e358682f9b8c407f6cd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd 2023-03-23 08:53:57,301 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd, entries=1, sequenceid=25, filesize=4.9 K 2023-03-23 08:53:57,303 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 465ms, sequenceid=25, compaction requested=true 2023-03-23 08:53:57,303 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:57,303 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:57,303 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:57,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:57,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:57,304 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:57,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,307 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,308 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,308 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:57,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:57,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f7d1170[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:57,314 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:57,314 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,315 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:57,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:57,315 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:57,316 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,316 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:57,318 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,318 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:57,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,318 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:57,318 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:57,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:57,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:57,323 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:57,324 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:57,324 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:57,324 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,324 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:57,334 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,334 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,335 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:57,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,336 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,338 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,342 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:57,342 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:57,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:57,342 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:57,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:57,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:57,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:57,342 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:57,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:57,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,444 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:57,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:57,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:57,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:57,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,464 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:57,464 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:57,465 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:57,465 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:57,465 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,465 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,467 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,467 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:57,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:57,467 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,467 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,468 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,468 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,469 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:57,470 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:57,470 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:57,470 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:57,470 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:57,470 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:57,470 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:57,487 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101cd9dc2f933f044caa17358b945ef6ee2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101cd9dc2f933f044caa17358b945ef6ee2 2023-03-23 08:53:57,894 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/42e0588cbb0b46afa4480c9c3bc62e42 2023-03-23 08:53:57,901 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/42e0588cbb0b46afa4480c9c3bc62e42 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42 2023-03-23 08:53:57,907 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42, entries=1, sequenceid=29, filesize=4.9 K 2023-03-23 08:53:57,909 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 438ms, sequenceid=29, compaction requested=true 2023-03-23 08:53:57,909 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:57,909 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:57,909 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:57,911 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,911 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,911 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,911 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:57,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57aaa75[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:57,913 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:57,913 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,916 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:57,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:57,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:57,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:57,918 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:57,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:57,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:57,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:57,922 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:57,922 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:57,922 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:57,922 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,922 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:57,924 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,924 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:57,924 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:57,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:57,925 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:57,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,929 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:57,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:57,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:57,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:57,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:57,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:57,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:57,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:57,935 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:57,935 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:57,936 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:57,937 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:58,037 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:58,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:58,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:58,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:58,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,049 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:58,049 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:58,050 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:58,050 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:58,050 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,050 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,052 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:58,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:58,053 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:58,053 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,053 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:58,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:58,059 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:58,059 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:58,060 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:58,060 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:58,082 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ddf8e12172dc424fbbf885058d25f509 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103ddf8e12172dc424fbbf885058d25f509 2023-03-23 08:53:58,091 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/0215f6ca93c54dfb814e88cf52b5895a 2023-03-23 08:53:58,097 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/0215f6ca93c54dfb814e88cf52b5895a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a 2023-03-23 08:53:58,107 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a, entries=1, sequenceid=33, filesize=4.9 K 2023-03-23 08:53:58,108 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 48ms, sequenceid=33, compaction requested=true 2023-03-23 08:53:58,109 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:58,109 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:58,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:58,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:58,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:58,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:58,111 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,111 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,111 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:58,112 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:58,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@df7b097[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:58,118 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:58,118 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,119 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,119 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:58,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:58,119 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:58,120 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:58,124 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:58,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:58,125 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:58,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:58,131 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:58,131 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:58,131 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:58,131 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,131 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:58,132 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,132 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,133 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:58,133 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,136 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:58,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:58,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:58,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:58,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:58,139 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:58,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:58,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:58,140 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:58,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:58,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:58,240 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:58,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:58,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:58,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:58,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,251 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:58,251 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:58,251 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:58,251 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:58,252 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,252 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:58,259 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:58,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:58,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:58,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:58,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:58,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,264 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:58,264 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:58,264 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:58,264 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:58,264 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:58,264 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:58,265 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:58,265 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:58,282 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075d0f2b03ff9e4f4aa89bf17effe544d5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509075d0f2b03ff9e4f4aa89bf17effe544d5 2023-03-23 08:53:58,466 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:53:58,694 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526 2023-03-23 08:53:58,710 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526 2023-03-23 08:53:58,717 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526, entries=1, sequenceid=37, filesize=4.9 K 2023-03-23 08:53:58,719 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 454ms, sequenceid=37, compaction requested=true 2023-03-23 08:53:58,719 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:58,719 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:58,719 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:58,719 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:58,719 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,719 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:58,719 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:58,753 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,753 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,753 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,753 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:58,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@130e74ec[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:58,755 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:58,755 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,768 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,768 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:58,768 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:58,769 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:58,770 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,770 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:58,808 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:58,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:58,812 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:58,812 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:58,812 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:58,812 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,812 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:58,861 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,861 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:58,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,861 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:58,861 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,864 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:58,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:58,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:58,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:58,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:58,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:58,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:53:58,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:58,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:58,900 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:58,901 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:58,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,901 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:58,901 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,901 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:58,901 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,001 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:59,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:53:59,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:53:59,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,011 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:59,011 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:59,011 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:53:59,011 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:53:59,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,045 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:59,045 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:59,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:59,046 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,046 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:53:59,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:53:59,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,047 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:53:59,047 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:53:59,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:53:59,049 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:53:59,049 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:53:59,049 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:53:59,049 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:59,049 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. started... 2023-03-23 08:53:59,049 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bdaddcf1cd39563f63add7141876dbbe 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:53:59,065 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120fa30b8d9a55243dfa63c44fe9528d8db to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fa30b8d9a55243dfa63c44fe9528d8db 2023-03-23 08:53:59,073 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/253e968e3b6147389b3176c864a96a9c 2023-03-23 08:53:59,078 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/253e968e3b6147389b3176c864a96a9c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c 2023-03-23 08:53:59,084 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c, entries=1, sequenceid=41, filesize=4.9 K 2023-03-23 08:53:59,085 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for bdaddcf1cd39563f63add7141876dbbe in 36ms, sequenceid=41, compaction requested=true 2023-03-23 08:53:59,085 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:59,085 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:59,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:53:59,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:53:59,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:53:59,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:53:59,138 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:53:59,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:59,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:53:59,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@244f016c[Count = 0] remaining members to acquire global barrier 2023-03-23 08:53:59,140 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:53:59,140 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,176 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,176 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:53:59,176 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:53:59,176 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:53:59,176 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,176 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:53:59,215 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:53:59,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:59,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:59,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:59,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:53:59,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:59,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:59,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:59,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:53:59,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:53:59,218 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:53:59,218 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:53:59,218 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:53:59,219 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,219 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:53:59,260 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,260 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:53:59,261 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:53:59,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:59,261 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:59,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,263 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:53:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:53:59,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,322 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:53:59,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:53:59,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-23 08:53:59,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:53:59,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:53:59,322 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:53:59,324 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:53:59,324 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,324 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,324 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:53:59,324 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:53:59,325 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:53:59,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:53:59,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:53:59,425 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:53:59,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:53:59,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-23 08:53:59,430 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:53:59,430 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:53:59,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:53:59,432 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562239432 2023-03-23 08:53:59,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:53:59,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-23 08:53:59,438 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:53:59,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x10ffb7cc to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:53:59,453 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27d3ac44, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:53:59,454 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:53:59,537 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d82eecadc7c0406c82361c07af0803b3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103d82eecadc7c0406c82361c07af0803b3 2023-03-23 08:53:59,539 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:53:59,555 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:53:59,557 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36174, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:53:59,571 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:53:59,572 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34376, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:53:59,594 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/37f899c5c08645a4bc371a6a54350557 first=Optional[r5] last=Optional[r6] 2023-03-23 08:53:59,595 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/37f899c5c08645a4bc371a6a54350557}] 2023-03-23 08:53:59,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/37f899c5c08645a4bc371a6a54350557 for inclusion in bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:53:59,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-23 08:53:59,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:53:59,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:59,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/37f899c5c08645a4bc371a6a54350557 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4t42r5ls210nom4b3lqgogrj2erqlcv3fsjsas0aqo3ffvjlq9stkoc8e233gg0o/family1/37f899c5c08645a4bc371a6a54350557 2023-03-23 08:53:59,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4t42r5ls210nom4b3lqgogrj2erqlcv3fsjsas0aqo3ffvjlq9stkoc8e233gg0o/family1/37f899c5c08645a4bc371a6a54350557 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_ 2023-03-23 08:53:59,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4t42r5ls210nom4b3lqgogrj2erqlcv3fsjsas0aqo3ffvjlq9stkoc8e233gg0o/family1/37f899c5c08645a4bc371a6a54350557 into bdaddcf1cd39563f63add7141876dbbe/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_ - updating store file list. 2023-03-23 08:53:59,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_ into bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:53:59,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4t42r5ls210nom4b3lqgogrj2erqlcv3fsjsas0aqo3ffvjlq9stkoc8e233gg0o/family1/37f899c5c08645a4bc371a6a54350557 into bdaddcf1cd39563f63add7141876dbbe/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_) 2023-03-23 08:53:59,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__4t42r5ls210nom4b3lqgogrj2erqlcv3fsjsas0aqo3ffvjlq9stkoc8e233gg0o/family1/37f899c5c08645a4bc371a6a54350557 2023-03-23 08:53:59,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:53:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:53:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): bdaddcf1cd39563f63add7141876dbbe/family1 is initiating minor compaction 2023-03-23 08:53:59,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@508dcb9c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:59,638 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bdaddcf1cd39563f63add7141876dbbe/family1 in testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:53:59,638 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp, totalSize=49.7 K 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 76a1965d3c234776b9127748274b9fb8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 1be50041ff0c450184dd6d0ff5a133c9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting bb9e74174fad417ab22c96a77a4347bf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting bdfddc9d82df46438fce2b22d67379e4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting dbb8804f01074e358682f9b8c407f6cd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 42e0588cbb0b46afa4480c9c3bc62e42, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 0215f6ca93c54dfb814e88cf52b5895a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 9ae9a3e04b7f4dceae3a0bbbf61f8526, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 253e968e3b6147389b3176c864a96a9c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:53:59,639 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:53:59,647 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:53:59,648 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:59,650 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101cd9dc2f933f044caa17358b945ef6ee2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101cd9dc2f933f044caa17358b945ef6ee2 2023-03-23 08:53:59,652 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103ddf8e12172dc424fbbf885058d25f509 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103ddf8e12172dc424fbbf885058d25f509 2023-03-23 08:53:59,652 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-23 08:53:59,652 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-23 08:53:59,653 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:53:59,653 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:53:59,685 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): bdaddcf1cd39563f63add7141876dbbe#family1#compaction#22 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:53:59,716 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/d5586bfa553248b0a12299f61afdaccd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/d5586bfa553248b0a12299f61afdaccd 2023-03-23 08:53:59,728 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in bdaddcf1cd39563f63add7141876dbbe/family1 of bdaddcf1cd39563f63add7141876dbbe into d5586bfa553248b0a12299f61afdaccd(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:53:59,728 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:53:59,728 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679561639638; duration=0sec 2023-03-23 08:53:59,728 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:53:59,836 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_] to archive 2023-03-23 08:53:59,837 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:53:59,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/76a1965d3c234776b9127748274b9fb8 2023-03-23 08:53:59,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1be50041ff0c450184dd6d0ff5a133c9 2023-03-23 08:53:59,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bb9e74174fad417ab22c96a77a4347bf 2023-03-23 08:53:59,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/bdfddc9d82df46438fce2b22d67379e4 2023-03-23 08:53:59,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/dbb8804f01074e358682f9b8c407f6cd 2023-03-23 08:53:59,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/42e0588cbb0b46afa4480c9c3bc62e42 2023-03-23 08:53:59,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0215f6ca93c54dfb814e88cf52b5895a 2023-03-23 08:53:59,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9ae9a3e04b7f4dceae3a0bbbf61f8526 2023-03-23 08:53:59,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/253e968e3b6147389b3176c864a96a9c 2023-03-23 08:53:59,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/0cc5c8eb266445e9ae6ba15fa9a4aad4_SeqId_44_ 2023-03-23 08:54:00,538 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a8bfccbbf5fa492fb043f2f7a8f95bfd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231a8bfccbbf5fa492fb043f2f7a8f95bfd 2023-03-23 08:54:00,540 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:54:00,574 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/3265a6577d4e46f999250f6beb95c2a6 first=Optional[r1] last=Optional[r4] 2023-03-23 08:54:00,575 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/3265a6577d4e46f999250f6beb95c2a6}] 2023-03-23 08:54:00,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/3265a6577d4e46f999250f6beb95c2a6 for inclusion in bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:54:00,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-23 08:54:00,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:00,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:54:00,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/3265a6577d4e46f999250f6beb95c2a6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__n02jc5flpgbk7480nvl23rtdbh9hpjhg1li1k5lrdrqh2j7di7d7aeepnn3m76sc/family1/3265a6577d4e46f999250f6beb95c2a6 2023-03-23 08:54:00,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__n02jc5flpgbk7480nvl23rtdbh9hpjhg1li1k5lrdrqh2j7di7d7aeepnn3m76sc/family1/3265a6577d4e46f999250f6beb95c2a6 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_ 2023-03-23 08:54:00,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__n02jc5flpgbk7480nvl23rtdbh9hpjhg1li1k5lrdrqh2j7di7d7aeepnn3m76sc/family1/3265a6577d4e46f999250f6beb95c2a6 into bdaddcf1cd39563f63add7141876dbbe/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_ - updating store file list. 2023-03-23 08:54:00,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_ into bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:54:00,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__n02jc5flpgbk7480nvl23rtdbh9hpjhg1li1k5lrdrqh2j7di7d7aeepnn3m76sc/family1/3265a6577d4e46f999250f6beb95c2a6 into bdaddcf1cd39563f63add7141876dbbe/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_) 2023-03-23 08:54:00,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__n02jc5flpgbk7480nvl23rtdbh9hpjhg1li1k5lrdrqh2j7di7d7aeepnn3m76sc/family1/3265a6577d4e46f999250f6beb95c2a6 2023-03-23 08:54:00,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:00,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:00,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): bdaddcf1cd39563f63add7141876dbbe/family1 is initiating minor compaction (all files) 2023-03-23 08:54:00,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2628b2aa; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:00,593 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bdaddcf1cd39563f63add7141876dbbe/family1 in testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:54:00,594 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/d5586bfa553248b0a12299f61afdaccd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp, totalSize=178.1 K 2023-03-23 08:54:00,594 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 80299271449940e6916dbe9a24f70463, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:54:00,595 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting d5586bfa553248b0a12299f61afdaccd, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:54:00,595 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 9dcbdb326288421e89e29301a1deb608_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241145 2023-03-23 08:54:00,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:54:00,598 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:00,599 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21bbb120d114290b7cb93cb0d4449df to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201e21bbb120d114290b7cb93cb0d4449df 2023-03-23 08:54:00,601 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512058944ef4e629c468e85827ab518142fab to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512058944ef4e629c468e85827ab518142fab 2023-03-23 08:54:00,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228f36294972265400298843afd9206c5e9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228f36294972265400298843afd9206c5e9 2023-03-23 08:54:00,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231eb5157a502d54bc6b7197d600cfdd8ad to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231eb5157a502d54bc6b7197d600cfdd8ad 2023-03-23 08:54:00,605 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-23 08:54:00,605 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-23 08:54:00,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:54:00,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:54:00,639 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): bdaddcf1cd39563f63add7141876dbbe#family1#compaction#23 average throughput is 8.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:01,152 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113076a954022356437280fa5218ba450e12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113076a954022356437280fa5218ba450e12 2023-03-23 08:54:01,153 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:54:01,186 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a65af69098b4c1db0111120a129aabf first=Optional[r00] last=Optional[r8] 2023-03-23 08:54:01,188 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a65af69098b4c1db0111120a129aabf}] 2023-03-23 08:54:01,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a65af69098b4c1db0111120a129aabf for inclusion in bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:54:01,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-23 08:54:01,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:01,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:54:01,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/2a65af69098b4c1db0111120a129aabf to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u0stvvmo0jq6vda79j43cej9cro3icubig1hfitj45123bfg35q72adak5760568/family1/2a65af69098b4c1db0111120a129aabf 2023-03-23 08:54:01,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u0stvvmo0jq6vda79j43cej9cro3icubig1hfitj45123bfg35q72adak5760568/family1/2a65af69098b4c1db0111120a129aabf as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1eae682d60f74f4fa4eb1b70170a8f60_SeqId_49_ 2023-03-23 08:54:01,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u0stvvmo0jq6vda79j43cej9cro3icubig1hfitj45123bfg35q72adak5760568/family1/2a65af69098b4c1db0111120a129aabf into bdaddcf1cd39563f63add7141876dbbe/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1eae682d60f74f4fa4eb1b70170a8f60_SeqId_49_ - updating store file list. 2023-03-23 08:54:01,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1eae682d60f74f4fa4eb1b70170a8f60_SeqId_49_ into bdaddcf1cd39563f63add7141876dbbe/family1 2023-03-23 08:54:01,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u0stvvmo0jq6vda79j43cej9cro3icubig1hfitj45123bfg35q72adak5760568/family1/2a65af69098b4c1db0111120a129aabf into bdaddcf1cd39563f63add7141876dbbe/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/1eae682d60f74f4fa4eb1b70170a8f60_SeqId_49_) 2023-03-23 08:54:01,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithMonthlyPolicy__u0stvvmo0jq6vda79j43cej9cro3icubig1hfitj45123bfg35q72adak5760568/family1/2a65af69098b4c1db0111120a129aabf 2023-03-23 08:54:01,207 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:54:01,208 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:01,210 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fa30b8d9a55243dfa63c44fe9528d8db to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120fa30b8d9a55243dfa63c44fe9528d8db 2023-03-23 08:54:01,212 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bd754fd829d74db295a350306923ea12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128bd754fd829d74db295a350306923ea12 2023-03-23 08:54:01,213 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb1a569302db4b748892bd709013e054 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb1a569302db4b748892bd709013e054 2023-03-23 08:54:01,213 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-23 08:54:01,213 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-23 08:54:01,214 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:54:01,214 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:54:01,214 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:54:01,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:54:01,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:01,216 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:01,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.7880 sec 2023-03-23 08:54:01,331 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=263 (was 249) Potentially hanging thread: hconnection-0x4573b769-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55532 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55592 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55632 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55708 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1370277958_15 at /127.0.0.1:55644 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55656 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50154 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4573b769-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55666 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55680 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55522 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-454665439-172.31.14.131-1679561612282:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55496 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4573b769-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x10ffb7cc-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1370277958_15 at /127.0.0.1:55718 [Receiving block BP-454665439-172.31.14.131-1679561612282:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x10ffb7cc-SendThread(127.0.0.1:50031) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55616 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1b0aee48-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4573b769-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:50031@0x10ffb7cc sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/219019176.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4573b769-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50280 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55608 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55506 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a7a625f83f14401da41242157b5afda4 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ResponseProcessor for block BP-454665439-172.31.14.131-1679561612282:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4573b769-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/329a531fe2994ed782c9e8dbf8686e6f block BP-454665439-172.31.14.131-1679561612282:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: hconnection-0x4573b769-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:55558 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50106 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1284455763_15 at /127.0.0.1:50136 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=629 (was 596) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=443 (was 404) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=6155 (was 6239) 2023-03-23 08:54:01,346 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=263, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=443, ProcessCount=169, AvailableMemoryMB=6154 2023-03-23 08:54:01,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:54:01,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:01,352 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:01,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-23 08:54:01,353 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:01,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:01,355 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:01,356 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76 empty. 2023-03-23 08:54:01,357 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:01,357 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-23 08:54:01,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:01,510 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/.tmp/family1/329a531fe2994ed782c9e8dbf8686e6f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/329a531fe2994ed782c9e8dbf8686e6f 2023-03-23 08:54:01,519 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in bdaddcf1cd39563f63add7141876dbbe/family1 of bdaddcf1cd39563f63add7141876dbbe into 329a531fe2994ed782c9e8dbf8686e6f(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:01,519 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:54:01,519 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679561640593; duration=0sec 2023-03-23 08:54:01,520 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:01,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/d5586bfa553248b0a12299f61afdaccd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_] to archive 2023-03-23 08:54:01,530 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:01,533 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/80299271449940e6916dbe9a24f70463 2023-03-23 08:54:01,534 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/d5586bfa553248b0a12299f61afdaccd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/d5586bfa553248b0a12299f61afdaccd 2023-03-23 08:54:01,536 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/family1/9dcbdb326288421e89e29301a1deb608_SeqId_47_ 2023-03-23 08:54:01,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:01,772 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:01,773 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 94e66683fc31732d625f937ede51eb76, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:01,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 94e66683fc31732d625f937ede51eb76, disabling compactions & flushes 2023-03-23 08:54:02,189 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. after waiting 0 ms 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,189 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,189 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:02,192 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:02,193 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561642192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561642192"}]},"ts":"1679561642192"} 2023-03-23 08:54:02,194 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:54:02,195 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:02,195 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561642195"}]},"ts":"1679561642195"} 2023-03-23 08:54:02,197 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-23 08:54:02,203 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=94e66683fc31732d625f937ede51eb76, ASSIGN}] 2023-03-23 08:54:02,205 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=94e66683fc31732d625f937ede51eb76, ASSIGN 2023-03-23 08:54:02,206 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=94e66683fc31732d625f937ede51eb76, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:02,357 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=94e66683fc31732d625f937ede51eb76, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:02,357 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561642357"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561642357"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561642357"}]},"ts":"1679561642357"} 2023-03-23 08:54:02,359 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 94e66683fc31732d625f937ede51eb76, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:02,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:02,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 94e66683fc31732d625f937ede51eb76, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:02,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:02,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,516 INFO [StoreOpener-94e66683fc31732d625f937ede51eb76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,517 DEBUG [StoreOpener-94e66683fc31732d625f937ede51eb76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:02,517 DEBUG [StoreOpener-94e66683fc31732d625f937ede51eb76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:02,518 INFO [StoreOpener-94e66683fc31732d625f937ede51eb76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 94e66683fc31732d625f937ede51eb76 columnFamilyName family1 2023-03-23 08:54:02,519 INFO [StoreOpener-94e66683fc31732d625f937ede51eb76-1] regionserver.HStore(310): Store=94e66683fc31732d625f937ede51eb76/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:02,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,522 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 94e66683fc31732d625f937ede51eb76 2023-03-23 08:54:02,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:02,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 94e66683fc31732d625f937ede51eb76; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74254979, jitterRate=0.10648541152477264}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:02,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:02,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76., pid=25, masterSystemTime=1679561642511 2023-03-23 08:54:02,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:02,528 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=94e66683fc31732d625f937ede51eb76, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:02,528 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561642528"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561642528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561642528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561642528"}]},"ts":"1679561642528"} 2023-03-23 08:54:02,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-23 08:54:02,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 94e66683fc31732d625f937ede51eb76, server=jenkins-hbase4.apache.org,32923,1679561616941 in 170 msec 2023-03-23 08:54:02,533 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-23 08:54:02,533 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=94e66683fc31732d625f937ede51eb76, ASSIGN in 328 msec 2023-03-23 08:54:02,533 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:02,534 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561642533"}]},"ts":"1679561642533"} 2023-03-23 08:54:02,535 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-23 08:54:02,537 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:02,539 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1880 sec 2023-03-23 08:54:02,592 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:03,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-23 08:54:03,460 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-23 08:54:03,483 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:03,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,484 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:03,484 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:03,484 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:03,484 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:03,485 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,485 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,486 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:03,486 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:03,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:03,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:03,486 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:03,486 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:03,488 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:03,488 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:03,488 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:03,488 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:03,489 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:54:03,912 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cf164ca2d8c74ef6965057399e98ea04 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130cf164ca2d8c74ef6965057399e98ea04 2023-03-23 08:54:04,316 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/186aca1219a24eb4b8a0402cd2848315 2023-03-23 08:54:04,322 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/186aca1219a24eb4b8a0402cd2848315 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315 2023-03-23 08:54:04,327 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-23 08:54:04,328 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 840ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:04,329 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:04,329 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:04,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:04,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:04,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:04,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:04,331 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74b62e92[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:04,332 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:04,332 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,333 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:04,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:04,334 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,334 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:04,336 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:04,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:04,338 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:04,338 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:04,338 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:04,338 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,338 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:04,340 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,340 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:04,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,340 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:04,340 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,341 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,343 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,343 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:04,343 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,343 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:04,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:04,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:04,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:04,343 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:04,343 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:04,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,344 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,444 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:04,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:04,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:04,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:04,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,451 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:04,452 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:04,452 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:04,452 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:04,452 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,452 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,454 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,454 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:04,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:04,454 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:04,454 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:04,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:04,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:04,456 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:04,456 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:04,456 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:54:04,870 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284715c31c420c47bdbc381a350e1b0d52 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511284715c31c420c47bdbc381a350e1b0d52 2023-03-23 08:54:05,277 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/46726b23304a4d458123e091b1b0baf3 2023-03-23 08:54:05,282 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/46726b23304a4d458123e091b1b0baf3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3 2023-03-23 08:54:05,287 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:54:05,287 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 831ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:05,287 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:05,288 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:05,289 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,289 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,290 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:05,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:05,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44f54b9b[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:05,291 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:05,291 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,293 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,293 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:05,293 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:05,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:05,293 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:05,295 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,295 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,295 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:05,295 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:05,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:05,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:05,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:05,297 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:05,297 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:05,297 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:05,297 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,297 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:05,298 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,298 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:05,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,299 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:05,299 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,300 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:05,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:05,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:05,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:05,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:05,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,302 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:05,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:05,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:05,303 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:05,403 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:05,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:05,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:05,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:05,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,410 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:05,410 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:05,410 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:05,410 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:05,410 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,410 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,413 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,413 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:05,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:05,413 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:05,413 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,414 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:05,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:05,414 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:05,414 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:05,415 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:05,828 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010150e65103db4cb881c1691c82d21c43 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512010150e65103db4cb881c1691c82d21c43 2023-03-23 08:54:06,234 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/236742d793794c37b559fb22340a5b5f 2023-03-23 08:54:06,240 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/236742d793794c37b559fb22340a5b5f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f 2023-03-23 08:54:06,244 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f, entries=1, sequenceid=13, filesize=4.9 K 2023-03-23 08:54:06,245 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 830ms, sequenceid=13, compaction requested=true 2023-03-23 08:54:06,245 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:06,245 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:06,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:06,246 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:06,246 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,246 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:06,246 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:06,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,247 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:06,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:06,248 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,248 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:06,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:06,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24bceefd[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:06,249 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:06,249 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,250 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:06,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:06,250 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,250 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:06,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:06,252 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,252 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:06,252 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,252 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:06,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:06,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:06,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:06,254 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:06,254 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:06,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:06,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:06,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,254 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:06,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:06,256 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:06,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,257 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:06,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:06,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,265 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,265 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,265 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:06,266 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:06,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:06,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:06,266 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:06,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:06,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:06,366 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:06,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:06,371 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:06,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:06,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,373 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:06,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:06,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:06,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:06,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,375 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,375 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:06,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:06,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:06,375 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:06,375 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:06,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:06,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,376 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,376 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:06,376 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:06,377 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:06,377 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:06,377 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:06,377 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:06,377 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:06,791 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aa1f0c631fe245ab850944ae89dcbd49 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa1f0c631fe245ab850944ae89dcbd49 2023-03-23 08:54:07,197 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/5b35abe9bddb446ab158e295542cde2d 2023-03-23 08:54:07,203 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/5b35abe9bddb446ab158e295542cde2d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d 2023-03-23 08:54:07,207 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d, entries=1, sequenceid=17, filesize=4.9 K 2023-03-23 08:54:07,208 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 831ms, sequenceid=17, compaction requested=true 2023-03-23 08:54:07,208 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:07,208 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:07,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:07,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:07,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,209 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:07,209 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:07,210 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,210 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:07,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:07,210 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,210 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:07,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:07,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:07,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:07,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:07,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78a4cf67[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:07,212 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:07,212 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,213 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:07,213 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,213 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:07,215 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:07,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:07,217 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:07,217 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:07,217 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:07,217 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,217 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:07,219 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,219 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:07,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,219 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:07,219 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,220 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,222 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:07,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:07,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:07,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:07,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:07,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:07,223 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:07,224 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,324 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:07,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:07,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:07,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:07,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,331 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:07,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:07,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:07,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:07,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,333 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:07,333 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:07,333 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:07,333 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:07,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:07,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:07,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:07,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:07,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:07,335 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:07,335 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:07,335 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:07,335 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:07,335 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:07,335 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:07,748 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228fea2074b7b4344c0af1d34801cc014cb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228fea2074b7b4344c0af1d34801cc014cb 2023-03-23 08:54:08,016 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:54:08,154 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/e06ef019950a44fabd710b2853fd7400 2023-03-23 08:54:08,160 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/e06ef019950a44fabd710b2853fd7400 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400 2023-03-23 08:54:08,165 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400, entries=1, sequenceid=21, filesize=4.9 K 2023-03-23 08:54:08,166 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 831ms, sequenceid=21, compaction requested=true 2023-03-23 08:54:08,166 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:08,166 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:08,171 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,171 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:08,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:08,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:08,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:08,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d23bc5f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:08,173 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:08,173 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,175 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:08,175 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,175 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:08,177 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,177 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:08,177 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:08,177 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:08,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:08,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:08,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:08,179 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:08,179 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:08,179 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:08,179 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,179 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:08,180 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,180 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,180 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:08,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:08,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:08,181 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:08,182 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:08,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:08,185 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:08,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:08,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:08,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,185 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:08,186 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:08,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:08,186 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:08,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:08,286 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:08,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:08,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:08,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:08,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,293 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:08,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:08,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:08,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:08,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,293 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,295 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:08,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:08,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:08,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:08,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:08,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:08,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:08,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:08,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:08,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:08,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:08,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:08,296 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:08,296 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:08,296 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:08,710 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231070d915064394cdf8919a7994d42c64e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231070d915064394cdf8919a7994d42c64e 2023-03-23 08:54:09,116 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/b1d5d9658e4f4cd18fe85f274cc094c2 2023-03-23 08:54:09,121 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/b1d5d9658e4f4cd18fe85f274cc094c2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2 2023-03-23 08:54:09,126 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2, entries=1, sequenceid=25, filesize=4.9 K 2023-03-23 08:54:09,127 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 831ms, sequenceid=25, compaction requested=true 2023-03-23 08:54:09,127 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:09,127 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:09,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:09,129 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,129 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,129 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,129 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:09,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53e481f3[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:09,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:09,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,132 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,132 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:09,132 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:09,132 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,132 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:09,132 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:09,134 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,134 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:09,134 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:09,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,134 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:09,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:09,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:09,136 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:09,136 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:09,136 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:09,136 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,136 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:09,142 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,142 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,143 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,143 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:09,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:09,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:09,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:09,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:09,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:09,146 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:09,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:09,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:09,147 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,247 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:09,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:09,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:09,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:09,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,254 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:09,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:09,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:09,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:09,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,254 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,258 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:09,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:09,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:09,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,259 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:09,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,259 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:09,259 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:09,259 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:09,261 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:09,261 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:09,261 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:09,261 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:09,261 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:09,675 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010173791dd0127b4f939006f0695fa5b396 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010173791dd0127b4f939006f0695fa5b396 2023-03-23 08:54:09,684 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/79a8f215db5146c98a31ee8041c270e0 2023-03-23 08:54:09,689 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/79a8f215db5146c98a31ee8041c270e0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0 2023-03-23 08:54:09,694 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0, entries=1, sequenceid=29, filesize=4.9 K 2023-03-23 08:54:09,695 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 434ms, sequenceid=29, compaction requested=true 2023-03-23 08:54:09,695 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:09,695 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:09,696 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,696 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10ec4674[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:09,698 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:09,698 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:09,699 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,699 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:09,701 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,701 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,701 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:09,702 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:09,703 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:09,704 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:09,704 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:09,704 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,704 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:09,705 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,705 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,705 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:09,706 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:09,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,707 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:09,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:09,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:09,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:09,709 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:09,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:09,709 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,710 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,810 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:09,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:09,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:09,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:09,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,817 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:09,817 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:09,817 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:09,817 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:09,817 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,817 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,818 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,818 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:09,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:09,819 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:09,819 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:09,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,820 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:09,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:09,820 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:09,820 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:09,820 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:09,820 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:09,820 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:09,820 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:09,820 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:09,821 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:10,234 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b0548dd4f7d4443fba3f4ae91276ae0a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0548dd4f7d4443fba3f4ae91276ae0a 2023-03-23 08:54:10,641 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/f2a58ee8c8dc46fbab679af6bb52cf79 2023-03-23 08:54:10,646 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/f2a58ee8c8dc46fbab679af6bb52cf79 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79 2023-03-23 08:54:10,650 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79, entries=1, sequenceid=33, filesize=4.9 K 2023-03-23 08:54:10,651 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 830ms, sequenceid=33, compaction requested=true 2023-03-23 08:54:10,651 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:10,651 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:10,651 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:10,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:10,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:10,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:10,653 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,653 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,654 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:10,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e6d7cf9[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:10,655 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:10,655 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,656 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:10,657 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,657 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:10,657 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:10,658 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:10,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:10,658 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:10,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:10,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:10,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:10,661 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:10,661 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:10,661 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:10,661 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,661 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:10,663 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,663 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,664 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:10,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:10,664 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,665 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,667 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,667 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:10,667 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:10,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:10,667 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:10,667 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:10,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:10,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:10,667 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:10,668 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:10,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:10,768 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:10,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:10,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:10,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:10,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,775 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:10,775 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:10,775 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:10,775 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:10,776 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,776 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,777 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,777 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:10,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:10,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:10,778 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:10,778 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:10,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:10,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,779 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:10,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:10,779 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:10,779 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:10,779 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:10,779 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:11,194 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b5a9f0228abb4ff4ad9eded3d7e4e6db to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907b5a9f0228abb4ff4ad9eded3d7e4e6db 2023-03-23 08:54:11,599 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/2d4c09ca1c344a42aaba4797d579c9f6 2023-03-23 08:54:11,605 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/2d4c09ca1c344a42aaba4797d579c9f6 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6 2023-03-23 08:54:11,610 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6, entries=1, sequenceid=37, filesize=4.9 K 2023-03-23 08:54:11,610 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 831ms, sequenceid=37, compaction requested=true 2023-03-23 08:54:11,611 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:11,611 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:11,611 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:11,611 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:11,611 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,611 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:11,611 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:11,613 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,613 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:11,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:11,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:11,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:11,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:11,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e8ce02f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:11,615 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:11,615 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,624 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:11,624 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,624 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:11,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:11,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:11,627 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,627 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:11,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:11,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:11,627 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:11,627 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:11,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:11,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:11,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:11,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:11,629 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:11,629 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:11,629 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:11,629 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:11,629 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,629 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:11,631 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,631 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:11,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:11,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,631 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:11,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,632 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:11,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:11,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:11,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:11,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,633 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:11,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:11,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:11,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:11,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:11,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:11,635 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:11,636 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:11,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:11,635 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,636 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,636 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,636 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:11,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:11,736 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:11,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:11,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:11,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:11,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,742 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:11,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:11,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:11,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:11,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,745 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:11,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:11,746 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:11,746 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,747 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:11,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:11,747 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:11,747 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:11,747 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:11,747 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:11,747 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:11,747 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:11,747 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. started... 2023-03-23 08:54:11,747 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 94e66683fc31732d625f937ede51eb76 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:12,163 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a94517187c234e01a6aa1a9893b0421f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120a94517187c234e01a6aa1a9893b0421f 2023-03-23 08:54:12,569 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/fcf6d575176b4178be8929eb07fda105 2023-03-23 08:54:12,574 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/fcf6d575176b4178be8929eb07fda105 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105 2023-03-23 08:54:12,579 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105, entries=1, sequenceid=41, filesize=4.9 K 2023-03-23 08:54:12,580 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 94e66683fc31732d625f937ede51eb76 in 833ms, sequenceid=41, compaction requested=true 2023-03-23 08:54:12,580 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:12,580 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:12,580 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:12,580 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:12,580 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,580 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:12,580 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:12,582 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,582 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:12,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:12,582 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:12,582 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:12,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:12,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19100f14[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:12,583 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:12,584 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,585 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:12,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:12,585 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,585 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:12,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:12,586 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:12,586 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,586 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:12,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:12,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:12,586 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:12,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:12,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:12,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:12,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:12,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:12,589 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:12,589 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:12,589 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:12,589 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,589 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:12,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:12,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:12,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:12,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:12,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:12,592 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:12,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:12,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,593 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:12,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:12,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:12,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-23 08:54:12,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,595 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:12,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:12,595 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,596 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:12,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:12,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:12,696 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:12,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:12,698 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-23 08:54:12,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:12,701 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:12,702 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:12,702 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562252702 2023-03-23 08:54:12,703 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:54:12,704 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-23 08:54:12,704 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:12,704 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x54efc075 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:12,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e64e720, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:12,709 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,551 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033d500f88c77c4ff3a860f6d8d9c6b83c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601033d500f88c77c4ff3a860f6d8d9c6b83c 2023-03-23 08:54:13,552 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:13,566 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:13,568 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38818, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:13,575 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:13,577 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40178, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:13,589 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/caacd39231d44190a4d6ff6e13dec328 first=Optional[r3] last=Optional[r6] 2023-03-23 08:54:13,590 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/caacd39231d44190a4d6ff6e13dec328}] 2023-03-23 08:54:13,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/caacd39231d44190a4d6ff6e13dec328 for inclusion in 94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:13,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-23 08:54:13,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:13,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:13,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/caacd39231d44190a4d6ff6e13dec328 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g1pik95cvkie1qq3l63a0a1gtai4uaidjkka099mom2h0trtvetn3ecfvoihuh0d/family1/caacd39231d44190a4d6ff6e13dec328 2023-03-23 08:54:13,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g1pik95cvkie1qq3l63a0a1gtai4uaidjkka099mom2h0trtvetn3ecfvoihuh0d/family1/caacd39231d44190a4d6ff6e13dec328 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_ 2023-03-23 08:54:13,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g1pik95cvkie1qq3l63a0a1gtai4uaidjkka099mom2h0trtvetn3ecfvoihuh0d/family1/caacd39231d44190a4d6ff6e13dec328 into 94e66683fc31732d625f937ede51eb76/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_ - updating store file list. 2023-03-23 08:54:13,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_ into 94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:13,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g1pik95cvkie1qq3l63a0a1gtai4uaidjkka099mom2h0trtvetn3ecfvoihuh0d/family1/caacd39231d44190a4d6ff6e13dec328 into 94e66683fc31732d625f937ede51eb76/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_) 2023-03-23 08:54:13,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g1pik95cvkie1qq3l63a0a1gtai4uaidjkka099mom2h0trtvetn3ecfvoihuh0d/family1/caacd39231d44190a4d6ff6e13dec328 2023-03-23 08:54:13,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:54:13,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:54:13,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 94e66683fc31732d625f937ede51eb76/family1 is initiating minor compaction 2023-03-23 08:54:13,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@170550b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:13,614 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 94e66683fc31732d625f937ede51eb76/family1 in testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:13,614 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp, totalSize=50.0 K 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 46726b23304a4d458123e091b1b0baf3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 236742d793794c37b559fb22340a5b5f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 5b35abe9bddb446ab158e295542cde2d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting e06ef019950a44fabd710b2853fd7400, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting b1d5d9658e4f4cd18fe85f274cc094c2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 79a8f215db5146c98a31ee8041c270e0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f2a58ee8c8dc46fbab679af6bb52cf79, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 2d4c09ca1c344a42aaba4797d579c9f6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting fcf6d575176b4178be8929eb07fda105, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:54:13,615 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting c6962dcf1943468a8f637acac53b4e10_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:54:13,616 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:13,617 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:13,619 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228fea2074b7b4344c0af1d34801cc014cb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228fea2074b7b4344c0af1d34801cc014cb 2023-03-23 08:54:13,621 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231070d915064394cdf8919a7994d42c64e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231070d915064394cdf8919a7994d42c64e 2023-03-23 08:54:13,626 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010173791dd0127b4f939006f0695fa5b396 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2016010173791dd0127b4f939006f0695fa5b396 2023-03-23 08:54:13,627 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0548dd4f7d4443fba3f4ae91276ae0a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103b0548dd4f7d4443fba3f4ae91276ae0a 2023-03-23 08:54:13,628 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-23 08:54:13,628 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:13,646 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 94e66683fc31732d625f937ede51eb76#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:14,120 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120554a9c9b6dee140f7bae1c12b3ef488af to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120554a9c9b6dee140f7bae1c12b3ef488af 2023-03-23 08:54:14,120 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/df8789e7dfe846c6b8a783adf5ed910f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/df8789e7dfe846c6b8a783adf5ed910f 2023-03-23 08:54:14,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:14,128 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 94e66683fc31732d625f937ede51eb76/family1 of 94e66683fc31732d625f937ede51eb76 into df8789e7dfe846c6b8a783adf5ed910f(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:14,128 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:14,129 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679561653614; duration=0sec 2023-03-23 08:54:14,129 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:14,169 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8ee89a2cb526488f92e33f78cab06e75 first=Optional[r1] last=Optional[r2] 2023-03-23 08:54:14,170 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8ee89a2cb526488f92e33f78cab06e75}] 2023-03-23 08:54:14,180 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8ee89a2cb526488f92e33f78cab06e75 for inclusion in 94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:14,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-23 08:54:14,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:14,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:14,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/8ee89a2cb526488f92e33f78cab06e75 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__romv4c5dhjiinhfrjht1a8e114b7vou9pe1dt773f7kln6tihbtvibd7rdccv49v/family1/8ee89a2cb526488f92e33f78cab06e75 2023-03-23 08:54:14,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__romv4c5dhjiinhfrjht1a8e114b7vou9pe1dt773f7kln6tihbtvibd7rdccv49v/family1/8ee89a2cb526488f92e33f78cab06e75 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_ 2023-03-23 08:54:14,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__romv4c5dhjiinhfrjht1a8e114b7vou9pe1dt773f7kln6tihbtvibd7rdccv49v/family1/8ee89a2cb526488f92e33f78cab06e75 into 94e66683fc31732d625f937ede51eb76/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_ - updating store file list. 2023-03-23 08:54:14,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_ into 94e66683fc31732d625f937ede51eb76/family1 2023-03-23 08:54:14,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__romv4c5dhjiinhfrjht1a8e114b7vou9pe1dt773f7kln6tihbtvibd7rdccv49v/family1/8ee89a2cb526488f92e33f78cab06e75 into 94e66683fc31732d625f937ede51eb76/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_) 2023-03-23 08:54:14,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithWeeklyPolicy__romv4c5dhjiinhfrjht1a8e114b7vou9pe1dt773f7kln6tihbtvibd7rdccv49v/family1/8ee89a2cb526488f92e33f78cab06e75 2023-03-23 08:54:14,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:14,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:14,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 94e66683fc31732d625f937ede51eb76/family1 is initiating minor compaction (all files) 2023-03-23 08:54:14,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5c4890a3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:14,206 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 94e66683fc31732d625f937ede51eb76/family1 in testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:54:14,206 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/df8789e7dfe846c6b8a783adf5ed910f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp, totalSize=176.8 K 2023-03-23 08:54:14,207 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 186aca1219a24eb4b8a0402cd2848315, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:54:14,207 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting df8789e7dfe846c6b8a783adf5ed910f, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:54:14,208 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 033107992a3d415b8f1b41221c0a8422_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241145 2023-03-23 08:54:14,211 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:14,212 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:14,215 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512010150e65103db4cb881c1691c82d21c43 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512010150e65103db4cb881c1691c82d21c43 2023-03-23 08:54:14,218 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa1f0c631fe245ab850944ae89dcbd49 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa1f0c631fe245ab850944ae89dcbd49 2023-03-23 08:54:14,218 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-23 08:54:14,218 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-23 08:54:14,220 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:14,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:14,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:14,221 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:14,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.5220 sec 2023-03-23 08:54:14,286 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 94e66683fc31732d625f937ede51eb76#family1#compaction#35 average throughput is 3.16 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:14,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_] to archive 2023-03-23 08:54:14,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:14,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/46726b23304a4d458123e091b1b0baf3 2023-03-23 08:54:14,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/236742d793794c37b559fb22340a5b5f 2023-03-23 08:54:14,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/5b35abe9bddb446ab158e295542cde2d 2023-03-23 08:54:14,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/e06ef019950a44fabd710b2853fd7400 2023-03-23 08:54:14,339 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-03-23 08:54:14,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/b1d5d9658e4f4cd18fe85f274cc094c2 2023-03-23 08:54:14,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/79a8f215db5146c98a31ee8041c270e0 2023-03-23 08:54:14,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/f2a58ee8c8dc46fbab679af6bb52cf79 2023-03-23 08:54:14,360 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/2d4c09ca1c344a42aaba4797d579c9f6 2023-03-23 08:54:14,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fcf6d575176b4178be8929eb07fda105 2023-03-23 08:54:14,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/c6962dcf1943468a8f637acac53b4e10_SeqId_44_ 2023-03-23 08:54:14,505 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256 (was 263), OpenFileDescriptor=622 (was 629), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=373 (was 443), ProcessCount=169 (was 169), AvailableMemoryMB=6093 (was 6154) 2023-03-23 08:54:14,519 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=256, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=373, ProcessCount=169, AvailableMemoryMB=6091 2023-03-23 08:54:14,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-23 08:54:14,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:14,533 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:14,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-23 08:54:14,534 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:14,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-23 08:54:14,537 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:14,537 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:14,537 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:14,537 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113 empty. 2023-03-23 08:54:14,537 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130 empty. 2023-03-23 08:54:14,538 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a empty. 2023-03-23 08:54:14,538 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:14,538 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:14,538 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:14,538 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-23 08:54:14,553 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:14,560 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2b2e5173dbc57f26ba388a87c96e3113, NAME => 'testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:14,561 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => be2d00f2bc48fa70033bcc38f2ad948a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:14,561 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => b39222917b68c0e24683b30b78e35130, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:14,596 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:14,596 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 2b2e5173dbc57f26ba388a87c96e3113, disabling compactions & flushes 2023-03-23 08:54:14,596 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:14,596 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:14,597 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. after waiting 0 ms 2023-03-23 08:54:14,597 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:14,597 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:14,597 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 2b2e5173dbc57f26ba388a87c96e3113: 2023-03-23 08:54:14,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:14,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing be2d00f2bc48fa70033bcc38f2ad948a, disabling compactions & flushes 2023-03-23 08:54:14,599 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:14,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:14,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. after waiting 0 ms 2023-03-23 08:54:14,599 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:14,599 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:14,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:14,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-23 08:54:14,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-23 08:54:14,846 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/.tmp/family1/fbfe3c021a4149f5b76b1a4c50fde305 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/fbfe3c021a4149f5b76b1a4c50fde305 2023-03-23 08:54:14,855 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 94e66683fc31732d625f937ede51eb76/family1 of 94e66683fc31732d625f937ede51eb76 into fbfe3c021a4149f5b76b1a4c50fde305(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:14,856 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:54:14,856 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1679561654206; duration=0sec 2023-03-23 08:54:14,856 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:14,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/df8789e7dfe846c6b8a783adf5ed910f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_] to archive 2023-03-23 08:54:14,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:14,930 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/186aca1219a24eb4b8a0402cd2848315 2023-03-23 08:54:14,932 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/df8789e7dfe846c6b8a783adf5ed910f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/df8789e7dfe846c6b8a783adf5ed910f 2023-03-23 08:54:14,934 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/family1/033107992a3d415b8f1b41221c0a8422_SeqId_47_ 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing b39222917b68c0e24683b30b78e35130, disabling compactions & flushes 2023-03-23 08:54:14,996 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. after waiting 0 ms 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:14,996 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:14,996 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for b39222917b68c0e24683b30b78e35130: 2023-03-23 08:54:15,004 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:15,007 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561655006"}]},"ts":"1679561655006"} 2023-03-23 08:54:15,007 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561655006"}]},"ts":"1679561655006"} 2023-03-23 08:54:15,007 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679561655006"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561655006"}]},"ts":"1679561655006"} 2023-03-23 08:54:15,017 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-23 08:54:15,020 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:15,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561655020"}]},"ts":"1679561655020"} 2023-03-23 08:54:15,023 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-23 08:54:15,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2b2e5173dbc57f26ba388a87c96e3113, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b39222917b68c0e24683b30b78e35130, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=be2d00f2bc48fa70033bcc38f2ad948a, ASSIGN}] 2023-03-23 08:54:15,031 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=be2d00f2bc48fa70033bcc38f2ad948a, ASSIGN 2023-03-23 08:54:15,031 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b39222917b68c0e24683b30b78e35130, ASSIGN 2023-03-23 08:54:15,031 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2b2e5173dbc57f26ba388a87c96e3113, ASSIGN 2023-03-23 08:54:15,032 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=be2d00f2bc48fa70033bcc38f2ad948a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:15,034 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b39222917b68c0e24683b30b78e35130, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:15,034 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2b2e5173dbc57f26ba388a87c96e3113, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:15,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-23 08:54:15,185 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b39222917b68c0e24683b30b78e35130, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,185 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679561655185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561655185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561655185"}]},"ts":"1679561655185"} 2023-03-23 08:54:15,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=2b2e5173dbc57f26ba388a87c96e3113, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655186"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561655186"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561655186"}]},"ts":"1679561655186"} 2023-03-23 08:54:15,186 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=be2d00f2bc48fa70033bcc38f2ad948a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,187 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655186"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561655186"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561655186"}]},"ts":"1679561655186"} 2023-03-23 08:54:15,189 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure b39222917b68c0e24683b30b78e35130, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:15,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 2b2e5173dbc57f26ba388a87c96e3113, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:15,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure be2d00f2bc48fa70033bcc38f2ad948a, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:15,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:15,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => be2d00f2bc48fa70033bcc38f2ad948a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.', STARTKEY => '2', ENDKEY => ''} 2023-03-23 08:54:15,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:15,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,353 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,355 DEBUG [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1 2023-03-23 08:54:15,355 DEBUG [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1 2023-03-23 08:54:15,356 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region be2d00f2bc48fa70033bcc38f2ad948a columnFamilyName family1 2023-03-23 08:54:15,360 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] regionserver.HStore(310): Store=be2d00f2bc48fa70033bcc38f2ad948a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,360 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,362 DEBUG [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family2 2023-03-23 08:54:15,362 DEBUG [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family2 2023-03-23 08:54:15,363 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region be2d00f2bc48fa70033bcc38f2ad948a columnFamilyName family2 2023-03-23 08:54:15,365 INFO [StoreOpener-be2d00f2bc48fa70033bcc38f2ad948a-1] regionserver.HStore(310): Store=be2d00f2bc48fa70033bcc38f2ad948a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:15,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:54:15,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:15,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened be2d00f2bc48fa70033bcc38f2ad948a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72987591, jitterRate=0.08759985864162445}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:15,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:15,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a., pid=33, masterSystemTime=1679561655342 2023-03-23 08:54:15,383 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=be2d00f2bc48fa70033bcc38f2ad948a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,384 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655383"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561655383"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561655383"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561655383"}]},"ts":"1679561655383"} 2023-03-23 08:54:15,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:15,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:15,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:15,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b39222917b68c0e24683b30b78e35130, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.', STARTKEY => '1', ENDKEY => '2'} 2023-03-23 08:54:15,388 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-03-23 08:54:15,388 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure be2d00f2bc48fa70033bcc38f2ad948a, server=jenkins-hbase4.apache.org,32923,1679561616941 in 194 msec 2023-03-23 08:54:15,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:15,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,390 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=be2d00f2bc48fa70033bcc38f2ad948a, ASSIGN in 360 msec 2023-03-23 08:54:15,416 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,419 DEBUG [StoreOpener-b39222917b68c0e24683b30b78e35130-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/family1 2023-03-23 08:54:15,419 DEBUG [StoreOpener-b39222917b68c0e24683b30b78e35130-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/family1 2023-03-23 08:54:15,420 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b39222917b68c0e24683b30b78e35130 columnFamilyName family1 2023-03-23 08:54:15,423 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] regionserver.HStore(310): Store=b39222917b68c0e24683b30b78e35130/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,423 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,428 DEBUG [StoreOpener-b39222917b68c0e24683b30b78e35130-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/family2 2023-03-23 08:54:15,428 DEBUG [StoreOpener-b39222917b68c0e24683b30b78e35130-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/family2 2023-03-23 08:54:15,429 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b39222917b68c0e24683b30b78e35130 columnFamilyName family2 2023-03-23 08:54:15,430 INFO [StoreOpener-b39222917b68c0e24683b30b78e35130-1] regionserver.HStore(310): Store=b39222917b68c0e24683b30b78e35130/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:15,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b39222917b68c0e24683b30b78e35130 2023-03-23 08:54:15,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:15,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b39222917b68c0e24683b30b78e35130; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65065465, jitterRate=-0.030449017882347107}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:15,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b39222917b68c0e24683b30b78e35130: 2023-03-23 08:54:15,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130., pid=31, masterSystemTime=1679561655342 2023-03-23 08:54:15,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:15,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:15,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:15,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b2e5173dbc57f26ba388a87c96e3113, NAME => 'testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.', STARTKEY => '', ENDKEY => '1'} 2023-03-23 08:54:15,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:15,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=b39222917b68c0e24683b30b78e35130, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679561655449"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561655449"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561655449"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561655449"}]},"ts":"1679561655449"} 2023-03-23 08:54:15,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=29 2023-03-23 08:54:15,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure b39222917b68c0e24683b30b78e35130, server=jenkins-hbase4.apache.org,32923,1679561616941 in 262 msec 2023-03-23 08:54:15,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=b39222917b68c0e24683b30b78e35130, ASSIGN in 425 msec 2023-03-23 08:54:15,467 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,479 DEBUG [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/family1 2023-03-23 08:54:15,479 DEBUG [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/family1 2023-03-23 08:54:15,480 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b2e5173dbc57f26ba388a87c96e3113 columnFamilyName family1 2023-03-23 08:54:15,484 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] regionserver.HStore(310): Store=2b2e5173dbc57f26ba388a87c96e3113/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,484 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,486 DEBUG [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/family2 2023-03-23 08:54:15,486 DEBUG [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/family2 2023-03-23 08:54:15,487 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b2e5173dbc57f26ba388a87c96e3113 columnFamilyName family2 2023-03-23 08:54:15,488 INFO [StoreOpener-2b2e5173dbc57f26ba388a87c96e3113-1] regionserver.HStore(310): Store=2b2e5173dbc57f26ba388a87c96e3113/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:15,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:15,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:54:15,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:15,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2b2e5173dbc57f26ba388a87c96e3113; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59923767, jitterRate=-0.10706628859043121}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:15,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2b2e5173dbc57f26ba388a87c96e3113: 2023-03-23 08:54:15,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113., pid=32, masterSystemTime=1679561655342 2023-03-23 08:54:15,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:15,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:15,517 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=2b2e5173dbc57f26ba388a87c96e3113, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,517 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679561655517"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561655517"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561655517"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561655517"}]},"ts":"1679561655517"} 2023-03-23 08:54:15,521 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-03-23 08:54:15,521 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 2b2e5173dbc57f26ba388a87c96e3113, server=jenkins-hbase4.apache.org,32923,1679561616941 in 328 msec 2023-03-23 08:54:15,524 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-03-23 08:54:15,524 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2b2e5173dbc57f26ba388a87c96e3113, ASSIGN in 493 msec 2023-03-23 08:54:15,525 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:15,525 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561655525"}]},"ts":"1679561655525"} 2023-03-23 08:54:15,528 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-23 08:54:15,530 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:15,532 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.0000 sec 2023-03-23 08:54:15,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-23 08:54:15,641 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-23 08:54:15,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:15,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,658 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-23 08:54:15,658 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:15,658 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-23 08:54:15,658 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-23 08:54:15,658 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,658 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,660 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:15,660 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:15,660 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:15,660 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:15,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:15,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-23 08:54:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-23 08:54:15,665 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:15,665 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. started... 2023-03-23 08:54:15,665 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:15,665 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. started... 2023-03-23 08:54:15,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:54:15,669 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:15,669 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b39222917b68c0e24683b30b78e35130: 2023-03-23 08:54:15,669 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:15,669 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. started... 2023-03-23 08:54:15,669 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing be2d00f2bc48fa70033bcc38f2ad948a 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-23 08:54:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:15,670 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:54:15,671 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2b2e5173dbc57f26ba388a87c96e3113: 2023-03-23 08:54:15,671 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:15,671 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:54:15,748 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303233e906c24311447398a377cbfd84b69a6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303233e906c24311447398a377cbfd84b69a6 2023-03-23 08:54:16,183 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp/family1/8c1212175bd84323a444111549b03a23 2023-03-23 08:54:16,189 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp/family1/8c1212175bd84323a444111549b03a23 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23 2023-03-23 08:54:16,196 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23, entries=1, sequenceid=5, filesize=4.9 K 2023-03-23 08:54:16,197 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for be2d00f2bc48fa70033bcc38f2ad948a in 528ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:16,197 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-23 08:54:16,198 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:16,198 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-23 08:54:16,198 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-23 08:54:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:16,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-23 08:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b3d25a4[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:16,205 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-23 08:54:16,205 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,208 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-23 08:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-23 08:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:16,208 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,208 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:16,211 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-23 08:54:16,212 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-23 08:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-23 08:54:16,219 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-23 08:54:16,219 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:16,219 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:16,219 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,219 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:16,221 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,221 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,221 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:16,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,221 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,235 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:16,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:16,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:16,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-23 08:54:16,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:16,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-23 08:54:16,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:16,236 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:16,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:16,236 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:16,336 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:16,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:16,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:16,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:16,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,345 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-23 08:54:16,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:16,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-23 08:54:16,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-23 08:54:16,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,347 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:16,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:16,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:16,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:16,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:16,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,348 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-23 08:54:16,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-23 08:54:16,348 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:16,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. started... 2023-03-23 08:54:16,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:54:16,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. started... 2023-03-23 08:54:16,349 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing be2d00f2bc48fa70033bcc38f2ad948a 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b39222917b68c0e24683b30b78e35130: 2023-03-23 08:54:16,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:16,350 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:54:16,350 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. started... 2023-03-23 08:54:16,350 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:54:16,350 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2b2e5173dbc57f26ba388a87c96e3113: 2023-03-23 08:54:16,350 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:54:16,350 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:54:16,369 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323f97c64024a3f4ff792bbf321f88ff3db to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230323f97c64024a3f4ff792bbf321f88ff3db 2023-03-23 08:54:16,781 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp/family1/f3f32279b5f347af95c71fd305ae7409 2023-03-23 08:54:16,787 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp/family1/f3f32279b5f347af95c71fd305ae7409 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409 2023-03-23 08:54:16,792 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409, entries=2, sequenceid=9, filesize=5.0 K 2023-03-23 08:54:16,795 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for be2d00f2bc48fa70033bcc38f2ad948a in 446ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:16,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:16,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-23 08:54:16,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-23 08:54:16,798 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,798 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,798 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,798 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:16,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-23 08:54:16,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e12168e[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:16,801 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-23 08:54:16,801 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,802 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-23 08:54:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-23 08:54:16,802 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:16,802 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,803 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-23 08:54:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:16,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-23 08:54:16,804 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:16,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-23 08:54:16,807 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-23 08:54:16,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:16,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:16,807 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,807 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:16,809 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,809 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,809 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:16,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:16,810 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:16,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:16,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,812 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:16,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:16,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:16,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:16,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:16,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-23 08:54:16,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:16,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-23 08:54:16,815 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,816 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,816 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,816 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:16,816 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:16,815 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:16,916 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:16,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:16,927 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-23 08:54:16,930 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-23 08:54:16,933 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-23 08:54:16,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:16,934 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562256934 2023-03-23 08:54:16,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:16,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-23 08:54:16,935 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-23 08:54:16,936 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x3f58e53d to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:16,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@76cc5811, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:16,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230323 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-23 08:54:16,994 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:16,995 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x54efc075 to 127.0.0.1:50031 2023-03-23 08:54:16,995 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:16,997 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:16,997 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10ffb7cc to 127.0.0.1:50031 2023-03-23 08:54:16,997 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:16,997 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:16,997 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x78ff69ed to 127.0.0.1:50031 2023-03-23 08:54:16,997 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:16,997 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:16,997 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c3492d1 to 127.0.0.1:50031 2023-03-23 08:54:16,997 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:17,850 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323d8f1e4b47a2b4f828d6fdbada8895b59 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230323d8f1e4b47a2b4f828d6fdbada8895b59 2023-03-23 08:54:17,851 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-23 08:54:17,871 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:17,872 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38824, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:17,881 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:17,883 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40184, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:17,901 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230323/family1/2c276fcfc82f4781965889c6f5002edc first=Optional[k0] last=Optional[k1] 2023-03-23 08:54:17,902 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230323/family1/2c276fcfc82f4781965889c6f5002edc}] 2023-03-23 08:54:17,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230323/family1/2c276fcfc82f4781965889c6f5002edc for inclusion in be2d00f2bc48fa70033bcc38f2ad948a/family1 2023-03-23 08:54:17,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-23 08:54:17,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-23 08:54:17,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:17,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230323/family1/2c276fcfc82f4781965889c6f5002edc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testScannerOnBulkLoadRefHFiles__irms87fmid80qcethgv1q0agofkcj0684jr939hp9vsjs91j3fk271cg3fj2uhsm/family1/2c276fcfc82f4781965889c6f5002edc 2023-03-23 08:54:17,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testScannerOnBulkLoadRefHFiles__irms87fmid80qcethgv1q0agofkcj0684jr939hp9vsjs91j3fk271cg3fj2uhsm/family1/2c276fcfc82f4781965889c6f5002edc as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_ 2023-03-23 08:54:17,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testScannerOnBulkLoadRefHFiles__irms87fmid80qcethgv1q0agofkcj0684jr939hp9vsjs91j3fk271cg3fj2uhsm/family1/2c276fcfc82f4781965889c6f5002edc into be2d00f2bc48fa70033bcc38f2ad948a/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_ - updating store file list. 2023-03-23 08:54:17,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_ into be2d00f2bc48fa70033bcc38f2ad948a/family1 2023-03-23 08:54:17,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testScannerOnBulkLoadRefHFiles__irms87fmid80qcethgv1q0agofkcj0684jr939hp9vsjs91j3fk271cg3fj2uhsm/family1/2c276fcfc82f4781965889c6f5002edc into be2d00f2bc48fa70033bcc38f2ad948a/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_) 2023-03-23 08:54:17,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testScannerOnBulkLoadRefHFiles__irms87fmid80qcethgv1q0agofkcj0684jr939hp9vsjs91j3fk271cg3fj2uhsm/family1/2c276fcfc82f4781965889c6f5002edc 2023-03-23 08:54:17,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:17,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:17,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): be2d00f2bc48fa70033bcc38f2ad948a/family1 is initiating minor compaction (all files) 2023-03-23 08:54:17,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6db363d5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:17,929 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of be2d00f2bc48fa70033bcc38f2ad948a/family1 in testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:54:17,929 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp, totalSize=15.1 K 2023-03-23 08:54:17,930 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 8c1212175bd84323a444111549b03a23, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561655641 2023-03-23 08:54:17,931 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f3f32279b5f347af95c71fd305ae7409, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561655641 2023-03-23 08:54:17,931 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting bf21dd34ba84460fa951fc8006aa8512_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561655641 2023-03-23 08:54:17,933 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-23 08:54:17,934 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:17,937 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303233e906c24311447398a377cbfd84b69a6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303233e906c24311447398a377cbfd84b69a6 2023-03-23 08:54:17,938 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230323f97c64024a3f4ff792bbf321f88ff3db to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230323f97c64024a3f4ff792bbf321f88ff3db 2023-03-23 08:54:17,938 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-23 08:54:17,938 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230323' 2023-03-23 08:54:17,939 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230323 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-23 08:54:17,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-23 08:54:17,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:17,940 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-23 08:54:17,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 1.0130 sec 2023-03-23 08:54:17,948 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): be2d00f2bc48fa70033bcc38f2ad948a#family1#compaction#38 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:17,981 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255 (was 256), OpenFileDescriptor=598 (was 622), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=373 (was 373), ProcessCount=169 (was 169), AvailableMemoryMB=5967 (was 6091) 2023-03-23 08:54:17,997 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=373, ProcessCount=169, AvailableMemoryMB=5967 2023-03-23 08:54:17,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:54:18,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:18,004 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:18,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-23 08:54:18,005 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:18,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:18,006 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:18,007 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b empty. 2023-03-23 08:54:18,007 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:18,008 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-23 08:54:18,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:18,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:18,419 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:18,420 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 80f02bad9c2288060a6ec1f940d1637b, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:18,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:18,773 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/.tmp/family1/b250866563d64a1a83825798aba9df56 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/b250866563d64a1a83825798aba9df56 2023-03-23 08:54:18,780 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in be2d00f2bc48fa70033bcc38f2ad948a/family1 of be2d00f2bc48fa70033bcc38f2ad948a into b250866563d64a1a83825798aba9df56(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:18,780 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:54:18,780 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1679561657929; duration=0sec 2023-03-23 08:54:18,780 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:18,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_] to archive 2023-03-23 08:54:18,824 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:18,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/8c1212175bd84323a444111549b03a23 2023-03-23 08:54:18,828 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/f3f32279b5f347af95c71fd305ae7409 2023-03-23 08:54:18,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/family1/bf21dd34ba84460fa951fc8006aa8512_SeqId_12_ 2023-03-23 08:54:18,831 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:18,831 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 80f02bad9c2288060a6ec1f940d1637b, disabling compactions & flushes 2023-03-23 08:54:18,831 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:18,832 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:18,832 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. after waiting 0 ms 2023-03-23 08:54:18,832 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:18,832 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:18,832 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:18,834 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:18,835 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561658835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561658835"}]},"ts":"1679561658835"} 2023-03-23 08:54:18,836 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:54:18,837 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:18,837 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561658837"}]},"ts":"1679561658837"} 2023-03-23 08:54:18,838 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-23 08:54:18,843 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=80f02bad9c2288060a6ec1f940d1637b, ASSIGN}] 2023-03-23 08:54:18,844 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=80f02bad9c2288060a6ec1f940d1637b, ASSIGN 2023-03-23 08:54:18,845 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=80f02bad9c2288060a6ec1f940d1637b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:18,996 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=80f02bad9c2288060a6ec1f940d1637b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:18,996 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561658996"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561658996"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561658996"}]},"ts":"1679561658996"} 2023-03-23 08:54:18,998 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 80f02bad9c2288060a6ec1f940d1637b, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:19,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:19,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:19,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 80f02bad9c2288060a6ec1f940d1637b, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:19,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:19,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,160 INFO [StoreOpener-80f02bad9c2288060a6ec1f940d1637b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,162 DEBUG [StoreOpener-80f02bad9c2288060a6ec1f940d1637b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:19,162 DEBUG [StoreOpener-80f02bad9c2288060a6ec1f940d1637b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:19,163 INFO [StoreOpener-80f02bad9c2288060a6ec1f940d1637b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80f02bad9c2288060a6ec1f940d1637b columnFamilyName family1 2023-03-23 08:54:19,165 INFO [StoreOpener-80f02bad9c2288060a6ec1f940d1637b-1] regionserver.HStore(310): Store=80f02bad9c2288060a6ec1f940d1637b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:19,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:54:19,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:19,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 80f02bad9c2288060a6ec1f940d1637b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67785436, jitterRate=0.010081708431243896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:19,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:19,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b., pid=37, masterSystemTime=1679561659150 2023-03-23 08:54:19,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:19,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:19,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=80f02bad9c2288060a6ec1f940d1637b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:19,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679561659184"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561659184"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561659184"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561659184"}]},"ts":"1679561659184"} 2023-03-23 08:54:19,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-23 08:54:19,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 80f02bad9c2288060a6ec1f940d1637b, server=jenkins-hbase4.apache.org,32923,1679561616941 in 188 msec 2023-03-23 08:54:19,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-23 08:54:19,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=80f02bad9c2288060a6ec1f940d1637b, ASSIGN in 346 msec 2023-03-23 08:54:19,192 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:19,193 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561659192"}]},"ts":"1679561659192"} 2023-03-23 08:54:19,201 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-23 08:54:19,205 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:19,209 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.2060 sec 2023-03-23 08:54:20,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-23 08:54:20,112 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-23 08:54:20,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:20,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,144 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:20,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:20,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:20,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:20,145 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,145 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,146 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:20,146 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:20,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:20,147 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:20,147 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,148 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:20,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,149 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:20,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:20,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:20,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:20,152 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:20,152 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:20,153 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:54:20,586 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130476c4633430648b0aa72b8c38e12c41a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130476c4633430648b0aa72b8c38e12c41a 2023-03-23 08:54:20,876 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:54:20,917 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:20,990 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/e4304e332a3848e580daab68fcd0b65b 2023-03-23 08:54:20,996 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/e4304e332a3848e580daab68fcd0b65b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b 2023-03-23 08:54:21,001 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-23 08:54:21,002 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 848ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:21,002 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:21,002 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:21,002 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:21,002 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:21,002 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,002 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:21,002 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:21,004 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,004 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,005 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,005 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:21,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47161afb[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:21,006 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:21,007 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,008 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,008 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:21,008 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:21,008 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:21,008 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,009 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:21,011 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,011 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:21,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,011 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:21,011 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:21,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:21,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:21,014 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:21,014 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:21,014 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:21,014 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,014 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:21,015 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,015 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,015 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:21,016 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,017 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:21,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:21,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,020 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:21,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:21,020 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:21,021 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:21,121 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:21,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:21,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:21,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:21,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,128 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:21,128 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:21,128 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:21,128 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:21,128 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,128 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:21,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:21,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:21,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:21,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:21,132 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:21,132 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:21,132 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:54:21,547 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a51cbb0acddb4e9ca1a678fc45445c0e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128a51cbb0acddb4e9ca1a678fc45445c0e 2023-03-23 08:54:21,554 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/406e25d4146b4c33b27f861ae0028f0f 2023-03-23 08:54:21,563 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/406e25d4146b4c33b27f861ae0028f0f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f 2023-03-23 08:54:21,567 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:54:21,568 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 436ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:21,568 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:21,568 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:21,568 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:21,568 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:21,568 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:21,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:21,570 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,570 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,571 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,571 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:21,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:21,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e67d5a2[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:21,574 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:21,574 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,583 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:21,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:21,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:21,583 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,583 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:21,584 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:21,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:21,585 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:21,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:21,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:21,587 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:21,587 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:21,587 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:21,587 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,587 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:21,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,591 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:21,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:21,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:21,592 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:21,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,597 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:21,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,600 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:21,600 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:21,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:21,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,601 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:21,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:21,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:21,601 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:21,601 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:21,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:21,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:21,602 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:21,702 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:21,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:21,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:21,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:21,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,709 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:21,709 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:21,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:21,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:21,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,712 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:21,712 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:21,712 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:21,712 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:21,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:21,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,713 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:21,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:21,713 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:21,714 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:21,714 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:21,714 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:21,714 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:22,130 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120125f6a4284cb146f7bb3a977dae11ef12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f6a4284cb146f7bb3a977dae11ef12 2023-03-23 08:54:22,537 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/a900c52644634ee9b81723ecef8090f1 2023-03-23 08:54:22,543 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/a900c52644634ee9b81723ecef8090f1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1 2023-03-23 08:54:22,547 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1, entries=1, sequenceid=13, filesize=4.9 K 2023-03-23 08:54:22,548 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 834ms, sequenceid=13, compaction requested=true 2023-03-23 08:54:22,548 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:22,548 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:22,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:22,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:22,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:22,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:22,551 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,551 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:22,551 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:22,552 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cf5cbcf[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:22,553 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:22,553 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,554 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:22,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:22,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:22,554 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,554 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:22,556 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:22,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:22,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:22,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:22,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:22,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:22,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:22,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:22,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:22,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:22,558 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:22,558 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:22,558 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:22,558 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,558 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:22,560 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,560 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,560 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:22,560 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:22,562 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:22,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,565 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,565 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:22,565 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:22,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:22,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:22,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:22,565 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,565 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:22,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:22,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:22,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:22,566 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:22,666 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:22,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:22,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:22,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:22,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,673 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:22,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:22,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:22,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:22,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,675 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,675 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:22,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:22,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:22,676 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:22,676 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:22,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,676 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:22,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,677 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:22,677 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:22,677 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:22,677 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:22,677 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:22,677 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:22,677 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:22,677 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:22,677 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:22,677 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:22,697 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ed5898f4c2b24fe8afd7eee4ee42c7e0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205ed5898f4c2b24fe8afd7eee4ee42c7e0 2023-03-23 08:54:23,103 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64 2023-03-23 08:54:23,108 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64 2023-03-23 08:54:23,112 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64, entries=1, sequenceid=17, filesize=4.9 K 2023-03-23 08:54:23,113 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 436ms, sequenceid=17, compaction requested=true 2023-03-23 08:54:23,114 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:23,114 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:23,114 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:23,116 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:23,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:23,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20d10b9[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:23,118 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:23,118 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,120 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,120 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:23,120 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:23,120 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,121 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,122 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,122 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:23,125 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:23,125 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:23,125 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:23,125 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:23,125 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,125 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:23,128 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,128 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,128 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,129 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,134 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,137 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,137 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:23,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:23,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,138 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:23,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,238 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:23,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:23,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:23,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:23,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,245 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,245 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,245 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:23,245 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:23,245 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,245 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,248 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,248 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,248 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,248 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:23,249 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:23,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,249 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:23,249 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:23,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:23,250 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:23,250 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,250 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:23,250 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:23,267 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122857844c72741b467b8dd915d3b9de5235 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122857844c72741b467b8dd915d3b9de5235 2023-03-23 08:54:23,274 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/696002c0ad624e7a9bcd45dd9257e332 2023-03-23 08:54:23,279 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/696002c0ad624e7a9bcd45dd9257e332 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332 2023-03-23 08:54:23,284 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332, entries=1, sequenceid=21, filesize=4.9 K 2023-03-23 08:54:23,285 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 35ms, sequenceid=21, compaction requested=true 2023-03-23 08:54:23,285 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:23,285 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,285 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:23,285 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:23,285 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,285 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:23,286 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:23,290 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,290 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,290 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,291 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:23,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:23,292 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@753b03c1[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:23,292 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:23,292 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,294 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:23,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:23,294 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,294 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:23,294 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:23,295 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,295 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:23,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,295 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,296 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:23,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:23,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:23,298 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:23,298 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:23,298 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:23,298 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,298 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:23,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,302 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,303 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,304 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,306 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:23,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:23,307 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:23,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,407 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:23,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:23,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:23,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:23,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,414 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,414 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,414 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:23,414 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:23,415 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,415 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,417 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,417 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:23,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,417 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:23,418 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,418 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:23,418 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,418 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:23,418 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:23,418 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:23,418 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,418 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:23,418 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:23,432 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d25f3faf0ef4468ca31bd56d558314d0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231d25f3faf0ef4468ca31bd56d558314d0 2023-03-23 08:54:23,838 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/54f2990ee9954ea2995baa65015a4533 2023-03-23 08:54:23,843 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/54f2990ee9954ea2995baa65015a4533 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533 2023-03-23 08:54:23,847 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533, entries=1, sequenceid=25, filesize=4.9 K 2023-03-23 08:54:23,848 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 430ms, sequenceid=25, compaction requested=true 2023-03-23 08:54:23,848 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:23,848 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,848 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:23,848 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:23,849 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,849 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:23,849 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:23,850 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,850 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,850 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,851 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:23,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:23,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15910201[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:23,852 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:23,852 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,853 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,853 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:23,854 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:23,854 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,854 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:23,854 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:23,856 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,856 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:23,856 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,856 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:23,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:23,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:23,860 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:23,861 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:23,861 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:23,861 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,861 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:23,862 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,862 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:23,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,862 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:23,863 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:23,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,864 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:23,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:23,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:23,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:23,867 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:23,867 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,868 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:23,868 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,868 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,868 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,868 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:23,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:23,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:23,968 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:23,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:23,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:23,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:23,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,975 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:23,976 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,976 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:23,976 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:23,976 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,976 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,978 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:23,978 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:23,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:23,978 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:23,978 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:23,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:23,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:23,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:23,979 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:23,979 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:23,979 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:23,980 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:23,980 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:23,980 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:23,980 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:23,980 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:24,395 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101223a40ec56574786be431047b8f1a210 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101223a40ec56574786be431047b8f1a210 2023-03-23 08:54:24,808 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/f83b55e1c0c040d9887cd48e08549742 2023-03-23 08:54:24,814 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/f83b55e1c0c040d9887cd48e08549742 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742 2023-03-23 08:54:24,818 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742, entries=1, sequenceid=29, filesize=4.9 K 2023-03-23 08:54:24,819 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 839ms, sequenceid=29, compaction requested=true 2023-03-23 08:54:24,819 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:24,819 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:24,819 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:24,822 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,822 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:24,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:24,822 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,823 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:24,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:24,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:24,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:24,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:24,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@674818cf[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:24,824 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:24,824 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,826 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:24,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:24,827 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:24,827 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,827 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:24,829 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,829 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:24,829 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:24,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:24,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:24,829 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:24,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:24,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:24,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:24,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:24,832 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:24,832 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:24,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:24,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:24,832 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,832 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:24,835 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,835 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,835 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:24,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:24,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:24,836 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:24,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,837 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:24,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,842 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:24,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:24,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:24,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,842 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:24,843 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:24,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:24,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:24,943 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:24,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:24,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:24,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:24,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,950 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:24,950 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:24,950 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:24,950 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:24,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,952 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,952 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:24,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:24,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:24,952 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:24,953 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:24,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,953 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:24,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,954 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,954 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:24,954 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:24,954 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:24,954 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:24,954 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:24,954 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:24,954 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:24,954 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:24,954 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:24,954 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:24,977 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601033604a15a9451440ca3c63884401c0051 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033604a15a9451440ca3c63884401c0051 2023-03-23 08:54:25,386 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/2a557eff892745389e8441a084ce8ba6 2023-03-23 08:54:25,394 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/2a557eff892745389e8441a084ce8ba6 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6 2023-03-23 08:54:25,401 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6, entries=1, sequenceid=33, filesize=4.9 K 2023-03-23 08:54:25,402 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 448ms, sequenceid=33, compaction requested=true 2023-03-23 08:54:25,402 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:25,402 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:25,402 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:25,402 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:25,402 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,402 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:25,402 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:25,404 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,404 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,405 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,405 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:25,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:25,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d2a4447[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:25,412 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:25,412 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,413 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:25,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:25,414 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:25,414 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,414 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:25,419 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:25,419 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,419 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:25,419 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:25,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:25,422 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:25,422 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:25,423 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:25,423 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,423 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:25,437 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,437 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,437 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,438 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,439 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,442 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:25,442 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:25,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:25,442 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,442 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:25,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:25,442 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,443 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:25,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,452 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:25,543 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:25,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:25,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:25,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:25,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,551 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:25,552 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:25,552 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:25,552 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:25,552 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,552 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,555 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:25,555 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:25,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:25,555 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,555 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:25,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:25,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,556 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:25,556 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:25,556 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:25,557 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:25,557 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:25,557 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:25,557 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:25,557 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:25,575 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070eb9ee24456e4be4925bf56386a9538c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509070eb9ee24456e4be4925bf56386a9538c 2023-03-23 08:54:25,582 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/83eaaffa19dc4625bbd42c8ada3d0252 2023-03-23 08:54:25,587 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/83eaaffa19dc4625bbd42c8ada3d0252 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252 2023-03-23 08:54:25,593 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252, entries=1, sequenceid=37, filesize=4.9 K 2023-03-23 08:54:25,594 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 37ms, sequenceid=37, compaction requested=true 2023-03-23 08:54:25,594 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:25,594 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:25,594 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:25,594 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:25,594 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,594 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:25,594 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:25,596 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,596 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,596 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,596 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:25,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20a27b28[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:25,598 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:25,598 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,600 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:25,600 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:25,600 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:25,600 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,600 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:25,601 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:25,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:25,604 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:25,604 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:25,604 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:25,604 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,604 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:25,607 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,607 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,607 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,608 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,609 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,611 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,611 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,611 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,611 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:25,611 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:25,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,611 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:25,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,612 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,612 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:25,713 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:25,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:25,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:25,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:25,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,720 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:25,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:25,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-23 08:54:25,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-23 08:54:25,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,723 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:25,723 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:25,723 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,723 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,724 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-23 08:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-23 08:54:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-23 08:54:25,724 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:25,725 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:25,725 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. started... 2023-03-23 08:54:25,725 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 80f02bad9c2288060a6ec1f940d1637b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:25,739 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112072ce891d68394a5d8ee7bec796c623b1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112072ce891d68394a5d8ee7bec796c623b1 2023-03-23 08:54:25,744 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7 2023-03-23 08:54:25,749 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7 2023-03-23 08:54:25,754 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7, entries=1, sequenceid=41, filesize=4.9 K 2023-03-23 08:54:25,755 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 80f02bad9c2288060a6ec1f940d1637b in 30ms, sequenceid=41, compaction requested=true 2023-03-23 08:54:25,755 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:25,755 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-23 08:54:25,755 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-23 08:54:25,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-23 08:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3dbef437[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:25,761 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-23 08:54:25,761 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,762 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,763 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:25,763 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-23 08:54:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-23 08:54:25,763 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:25,766 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-23 08:54:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,766 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-23 08:54:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:25,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:54:25,768 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-23 08:54:25,768 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:25,768 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:25,768 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,768 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:25,770 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,770 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:25,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,770 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:25,770 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,772 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:25,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:25,779 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-23 08:54:25,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:25,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:25,779 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:25,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-23 08:54:25,780 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:25,880 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:25,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:25,882 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-23 08:54:25,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:25,885 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:25,890 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562265890 2023-03-23 08:54:25,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:25,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:25,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-23 08:54:25,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:25,896 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x782e5dab to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:25,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6cb68e6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:25,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:25,961 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b524134730144aa1ad98b3816856a97b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103b524134730144aa1ad98b3816856a97b 2023-03-23 08:54:25,962 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:26,004 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:26,006 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49576, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:26,029 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:26,030 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47158, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:26,046 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e first=Optional[r3] last=Optional[r6] 2023-03-23 08:54:26,046 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e}] 2023-03-23 08:54:26,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e for inclusion in 80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:26,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-23 08:54:26,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:26,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:26,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qb1cf96100jtp70nlq9nmqdnqvriue0469ag9fhrpno6it48e38rs7bqrm9m1ifk/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e 2023-03-23 08:54:26,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qb1cf96100jtp70nlq9nmqdnqvriue0469ag9fhrpno6it48e38rs7bqrm9m1ifk/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_ 2023-03-23 08:54:26,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qb1cf96100jtp70nlq9nmqdnqvriue0469ag9fhrpno6it48e38rs7bqrm9m1ifk/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e into 80f02bad9c2288060a6ec1f940d1637b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_ - updating store file list. 2023-03-23 08:54:26,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_ into 80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:26,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qb1cf96100jtp70nlq9nmqdnqvriue0469ag9fhrpno6it48e38rs7bqrm9m1ifk/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e into 80f02bad9c2288060a6ec1f940d1637b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_) 2023-03-23 08:54:26,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qb1cf96100jtp70nlq9nmqdnqvriue0469ag9fhrpno6it48e38rs7bqrm9m1ifk/family1/1b3cb81f0cd9486582ca4fbcb7a9db8e 2023-03-23 08:54:26,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:54:26,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:54:26,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 80f02bad9c2288060a6ec1f940d1637b/family1 is initiating minor compaction 2023-03-23 08:54:26,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@18066268; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:26,080 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 80f02bad9c2288060a6ec1f940d1637b/family1 in testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:26,080 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp, totalSize=50.0 K 2023-03-23 08:54:26,080 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 406e25d4146b4c33b27f861ae0028f0f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting a900c52644634ee9b81723ecef8090f1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting b3fc0cb1dfa64f7586e7dc6710f3cd64, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 696002c0ad624e7a9bcd45dd9257e332, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 54f2990ee9954ea2995baa65015a4533, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f83b55e1c0c040d9887cd48e08549742, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 2a557eff892745389e8441a084ce8ba6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 83eaaffa19dc4625bbd42c8ada3d0252, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 9e58cd5fab0f44c3a50be1ba0d07e9a7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:54:26,081 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:54:26,086 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:26,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:26,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122857844c72741b467b8dd915d3b9de5235 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122857844c72741b467b8dd915d3b9de5235 2023-03-23 08:54:26,091 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231d25f3faf0ef4468ca31bd56d558314d0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231d25f3faf0ef4468ca31bd56d558314d0 2023-03-23 08:54:26,093 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101223a40ec56574786be431047b8f1a210 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101223a40ec56574786be431047b8f1a210 2023-03-23 08:54:26,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033604a15a9451440ca3c63884401c0051 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601033604a15a9451440ca3c63884401c0051 2023-03-23 08:54:26,094 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-23 08:54:26,094 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,157 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80f02bad9c2288060a6ec1f940d1637b#family1#compaction#49 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:26,647 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055f184fcbb968421b8b44902898e328d3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512055f184fcbb968421b8b44902898e328d3 2023-03-23 08:54:26,647 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/33c65bcb75334c08ae12743ff7990129 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/33c65bcb75334c08ae12743ff7990129 2023-03-23 08:54:26,648 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:26,669 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 80f02bad9c2288060a6ec1f940d1637b/family1 of 80f02bad9c2288060a6ec1f940d1637b into 33c65bcb75334c08ae12743ff7990129(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:26,669 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:26,669 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679561666080; duration=0sec 2023-03-23 08:54:26,669 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:26,719 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/511ac682b452467b93e1916a650323c1 first=Optional[r00] last=Optional[r2] 2023-03-23 08:54:26,720 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/511ac682b452467b93e1916a650323c1}] 2023-03-23 08:54:26,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/511ac682b452467b93e1916a650323c1 for inclusion in 80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:26,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_] to archive 2023-03-23 08:54:26,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-23 08:54:26,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:26,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:26,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:26,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/511ac682b452467b93e1916a650323c1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fialflhfuodalct43ci0hqpe1348m3172d5baq1fmko81fvn4iec2badt21scrd1/family1/511ac682b452467b93e1916a650323c1 2023-03-23 08:54:26,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/406e25d4146b4c33b27f861ae0028f0f 2023-03-23 08:54:26,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fialflhfuodalct43ci0hqpe1348m3172d5baq1fmko81fvn4iec2badt21scrd1/family1/511ac682b452467b93e1916a650323c1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_ 2023-03-23 08:54:26,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fialflhfuodalct43ci0hqpe1348m3172d5baq1fmko81fvn4iec2badt21scrd1/family1/511ac682b452467b93e1916a650323c1 into 80f02bad9c2288060a6ec1f940d1637b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_ - updating store file list. 2023-03-23 08:54:26,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/a900c52644634ee9b81723ecef8090f1 2023-03-23 08:54:26,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/b3fc0cb1dfa64f7586e7dc6710f3cd64 2023-03-23 08:54:26,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/696002c0ad624e7a9bcd45dd9257e332 2023-03-23 08:54:26,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/54f2990ee9954ea2995baa65015a4533 2023-03-23 08:54:26,764 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f83b55e1c0c040d9887cd48e08549742 2023-03-23 08:54:26,766 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/2a557eff892745389e8441a084ce8ba6 2023-03-23 08:54:26,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/83eaaffa19dc4625bbd42c8ada3d0252 2023-03-23 08:54:26,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_ into 80f02bad9c2288060a6ec1f940d1637b/family1 2023-03-23 08:54:26,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fialflhfuodalct43ci0hqpe1348m3172d5baq1fmko81fvn4iec2badt21scrd1/family1/511ac682b452467b93e1916a650323c1 into 80f02bad9c2288060a6ec1f940d1637b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_) 2023-03-23 08:54:26,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fialflhfuodalct43ci0hqpe1348m3172d5baq1fmko81fvn4iec2badt21scrd1/family1/511ac682b452467b93e1916a650323c1 2023-03-23 08:54:26,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:26,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/9e58cd5fab0f44c3a50be1ba0d07e9a7 2023-03-23 08:54:26,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/f80c2466218f48bf866bd6558ee2b8f1_SeqId_44_ 2023-03-23 08:54:26,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:26,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 80f02bad9c2288060a6ec1f940d1637b/family1 is initiating minor compaction (all files) 2023-03-23 08:54:26,776 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 80f02bad9c2288060a6ec1f940d1637b/family1 in testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:54:26,776 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/33c65bcb75334c08ae12743ff7990129, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp, totalSize=337.0 K 2023-03-23 08:54:26,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@76b85852; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:26,776 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting e4304e332a3848e580daab68fcd0b65b, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:54:26,777 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 33c65bcb75334c08ae12743ff7990129, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:54:26,777 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting fa504e3c970c4e63af830f6341762566_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841145 2023-03-23 08:54:26,779 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:26,780 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:26,783 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130476c4633430648b0aa72b8c38e12c41a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130476c4633430648b0aa72b8c38e12c41a 2023-03-23 08:54:26,786 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f6a4284cb146f7bb3a977dae11ef12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120125f6a4284cb146f7bb3a977dae11ef12 2023-03-23 08:54:26,794 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205ed5898f4c2b24fe8afd7eee4ee42c7e0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205ed5898f4c2b24fe8afd7eee4ee42c7e0 2023-03-23 08:54:26,794 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-23 08:54:26,794 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-23 08:54:26,795 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-23 08:54:26,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-23 08:54:26,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:26,804 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:54:26,806 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 921 msec 2023-03-23 08:54:26,821 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 80f02bad9c2288060a6ec1f940d1637b#family1#compaction#50 average throughput is 4.69 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:26,954 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=261 (was 255) - Thread LEAK? -, OpenFileDescriptor=633 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=384 (was 373) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5931 (was 5967) 2023-03-23 08:54:26,971 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=261, OpenFileDescriptor=633, MaxFileDescriptor=60000, SystemLoadAverage=384, ProcessCount=169, AvailableMemoryMB=5930 2023-03-23 08:54:26,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:54:26,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:26,976 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:26,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-23 08:54:26,977 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:26,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-23 08:54:26,979 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:26,981 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 empty. 2023-03-23 08:54:26,981 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:26,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-23 08:54:27,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-23 08:54:27,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-23 08:54:27,398 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:27,401 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6567fac53213b6f4d692c06a6cbc7ae8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:27,416 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:27,416 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 6567fac53213b6f4d692c06a6cbc7ae8, disabling compactions & flushes 2023-03-23 08:54:27,416 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,416 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,416 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. after waiting 0 ms 2023-03-23 08:54:27,416 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,416 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,417 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:27,419 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:27,420 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561667420"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561667420"}]},"ts":"1679561667420"} 2023-03-23 08:54:27,421 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:54:27,422 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:27,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561667422"}]},"ts":"1679561667422"} 2023-03-23 08:54:27,423 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-23 08:54:27,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, ASSIGN}] 2023-03-23 08:54:27,429 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, ASSIGN 2023-03-23 08:54:27,430 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:27,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-23 08:54:27,582 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:27,582 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561667582"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561667582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561667582"}]},"ts":"1679561667582"} 2023-03-23 08:54:27,584 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:27,695 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/.tmp/family1/4327a5d502ba40f7887374b317d1c7ec as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/4327a5d502ba40f7887374b317d1c7ec 2023-03-23 08:54:27,700 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 80f02bad9c2288060a6ec1f940d1637b/family1 of 80f02bad9c2288060a6ec1f940d1637b into 4327a5d502ba40f7887374b317d1c7ec(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:27,700 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:54:27,701 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1679561666775; duration=0sec 2023-03-23 08:54:27,701 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:27,729 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/33c65bcb75334c08ae12743ff7990129, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_] to archive 2023-03-23 08:54:27,730 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:27,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/e4304e332a3848e580daab68fcd0b65b 2023-03-23 08:54:27,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/33c65bcb75334c08ae12743ff7990129 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/33c65bcb75334c08ae12743ff7990129 2023-03-23 08:54:27,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/family1/fa504e3c970c4e63af830f6341762566_SeqId_47_ 2023-03-23 08:54:27,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6567fac53213b6f4d692c06a6cbc7ae8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:27,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:27,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,741 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,743 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:27,743 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:27,743 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6567fac53213b6f4d692c06a6cbc7ae8 columnFamilyName family1 2023-03-23 08:54:27,745 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(310): Store=6567fac53213b6f4d692c06a6cbc7ae8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:27,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,749 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:27,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:27,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6567fac53213b6f4d692c06a6cbc7ae8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69730200, jitterRate=0.03906095027923584}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:27,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:27,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., pid=41, masterSystemTime=1679561667737 2023-03-23 08:54:27,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:27,755 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:27,755 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561667755"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561667755"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561667755"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561667755"}]},"ts":"1679561667755"} 2023-03-23 08:54:27,759 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-23 08:54:27,759 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941 in 173 msec 2023-03-23 08:54:27,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-23 08:54:27,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, ASSIGN in 331 msec 2023-03-23 08:54:27,761 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:27,761 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561667761"}]},"ts":"1679561667761"} 2023-03-23 08:54:27,763 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-23 08:54:27,766 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:27,767 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 793 msec 2023-03-23 08:54:28,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-23 08:54:28,083 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-23 08:54:28,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:28,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,134 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:28,134 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:28,134 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:28,134 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:28,134 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,135 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,136 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:28,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:28,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:28,136 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,137 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,137 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:28,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:28,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,140 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:28,140 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:28,140 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:28,144 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:28,144 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:28,144 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:28,145 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:54:28,200 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113009486e01eb7548fc89fbe87b2d07f935 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113009486e01eb7548fc89fbe87b2d07f935 2023-03-23 08:54:28,206 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/1172c073a7df45528aef3d8d14718473 2023-03-23 08:54:28,212 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/1172c073a7df45528aef3d8d14718473 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473 2023-03-23 08:54:28,218 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-23 08:54:28,219 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 74ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:28,219 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:28,220 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:28,220 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:28,220 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:28,220 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:28,220 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,220 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:28,220 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:28,221 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:28,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:28,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:28,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:28,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a170d2f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:28,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:28,224 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,225 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,225 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:28,225 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:28,225 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:28,225 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,225 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:28,227 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:28,227 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:28,227 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:28,227 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:28,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:28,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:28,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:28,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:28,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:28,230 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:28,230 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:28,230 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:28,230 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,230 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:28,232 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,232 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,232 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:28,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:28,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:28,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:28,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:28,232 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:28,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:28,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:28,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:28,237 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:28,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:28,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:28,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:28,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:28,337 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:28,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:28,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:28,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:28,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,345 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:28,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:28,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:28,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:28,346 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,346 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,347 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:28,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:28,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:28,347 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:28,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:28,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,348 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,349 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:28,349 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:28,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:28,349 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:28,349 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:54:28,767 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112802b06dd3901d4280bff8cc6467968cf2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112802b06dd3901d4280bff8cc6467968cf2 2023-03-23 08:54:29,177 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/fa85e5852ce44d55a238e9498138435d 2023-03-23 08:54:29,183 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/fa85e5852ce44d55a238e9498138435d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d 2023-03-23 08:54:29,189 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:54:29,190 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 841ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:29,190 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:29,190 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:29,190 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:29,190 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:29,190 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,190 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:29,190 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:29,192 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,192 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,192 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,192 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:29,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45c06a56[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:29,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:29,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,195 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:29,196 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,196 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:29,197 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:29,197 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,197 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:29,198 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:29,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:29,201 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:29,201 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:29,201 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:29,201 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,201 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:29,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,205 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,206 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:29,216 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,217 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,217 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,217 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:29,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:29,217 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:29,217 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:29,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,317 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:29,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:29,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:29,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:29,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,325 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:29,325 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:29,325 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:29,325 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:29,327 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,327 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,330 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:29,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:29,330 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,331 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:29,331 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,331 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,331 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,332 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:29,332 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:29,332 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:29,332 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:29,332 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:29,332 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:29,332 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:29,332 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:29,351 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014c80db130dd44f93aeea23b4c8621300 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014c80db130dd44f93aeea23b4c8621300 2023-03-23 08:54:29,356 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/7985e8493d3f425abc5c3c4131793812 2023-03-23 08:54:29,360 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/7985e8493d3f425abc5c3c4131793812 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812 2023-03-23 08:54:29,364 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812, entries=1, sequenceid=13, filesize=5.0 K 2023-03-23 08:54:29,366 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 34ms, sequenceid=13, compaction requested=true 2023-03-23 08:54:29,366 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:29,366 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:29,366 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:29,366 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:29,366 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,366 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:29,366 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:29,369 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,369 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,369 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,369 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:29,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@768822fd[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:29,371 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:29,371 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,372 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:29,372 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:29,372 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,373 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:29,375 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:29,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:29,378 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:29,378 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:29,378 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:29,378 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,378 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:29,380 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,380 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,380 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,380 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,382 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,388 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:29,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:29,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:29,389 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,389 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:29,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:29,388 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:29,389 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,489 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:29,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:29,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:29,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:29,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,497 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:29,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:29,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:29,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:29,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,499 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:29,499 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:29,499 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,499 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:29,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,501 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:29,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:29,501 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:29,502 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:29,502 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:29,502 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:29,502 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:29,502 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:29,520 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d7a1ff79ee89483cbac7f623be032fc0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d7a1ff79ee89483cbac7f623be032fc0 2023-03-23 08:54:29,931 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/d74ed6825dfc4781bd7592b86e29f015 2023-03-23 08:54:29,936 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/d74ed6825dfc4781bd7592b86e29f015 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015 2023-03-23 08:54:29,941 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015, entries=1, sequenceid=17, filesize=5.0 K 2023-03-23 08:54:29,942 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 440ms, sequenceid=17, compaction requested=true 2023-03-23 08:54:29,942 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:29,942 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:29,942 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:29,942 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:29,942 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,942 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:29,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:29,944 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,944 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,945 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:29,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cdd7f57[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:29,947 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:29,947 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,948 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,948 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:29,948 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:29,948 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:29,951 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:29,951 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,951 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:29,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:29,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:29,952 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:29,952 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:29,952 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:29,952 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,952 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:29,953 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,953 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,953 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:29,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,954 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:29,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,956 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:29,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:29,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:29,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:29,959 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:29,959 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:29,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:29,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:29,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:29,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:29,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,060 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:30,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:30,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:30,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:30,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,096 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:30,096 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,096 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:30,096 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:30,097 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,097 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,098 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:30,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:30,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:30,099 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,099 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,100 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:30,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:30,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,100 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,104 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:30,104 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,104 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:30,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:30,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:30,105 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:30,105 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:30,105 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:30,130 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c6e256d620e3492c82012d6b9a699a8c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c6e256d620e3492c82012d6b9a699a8c 2023-03-23 08:54:30,136 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/48b24ab7d87d44579e1f97a8e141ae1d 2023-03-23 08:54:30,142 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/48b24ab7d87d44579e1f97a8e141ae1d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d 2023-03-23 08:54:30,148 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d, entries=1, sequenceid=21, filesize=5.0 K 2023-03-23 08:54:30,149 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 44ms, sequenceid=21, compaction requested=true 2023-03-23 08:54:30,149 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:30,149 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:30,151 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,151 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,151 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,151 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e817396[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:30,155 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:30,155 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,160 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:30,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:30,161 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:30,162 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,162 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:30,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:30,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:30,163 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,163 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:30,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:30,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:30,165 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:30,165 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:30,165 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:30,165 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,165 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:30,167 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,167 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:30,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,168 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,169 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,170 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,174 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:30,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:30,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:30,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:30,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:30,174 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,175 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:30,175 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:30,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:30,175 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:30,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:30,275 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:30,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:30,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:30,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:30,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,282 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:30,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:30,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:30,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,285 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:30,285 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:30,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:30,285 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,285 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:30,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:30,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,286 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:30,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,286 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:30,286 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,286 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:30,287 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:30,287 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:30,287 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:30,287 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:30,288 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:30,301 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317bb2df8b09c04c309196c1f7d1534a25 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512317bb2df8b09c04c309196c1f7d1534a25 2023-03-23 08:54:30,718 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/f67044ebdc6d4172801e82c3d7bb86b0 2023-03-23 08:54:30,725 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/f67044ebdc6d4172801e82c3d7bb86b0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0 2023-03-23 08:54:30,730 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0, entries=1, sequenceid=25, filesize=5.0 K 2023-03-23 08:54:30,731 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 444ms, sequenceid=25, compaction requested=true 2023-03-23 08:54:30,732 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:30,732 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:30,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:30,734 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,734 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,734 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,734 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:30,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@585538ca[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:30,736 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:30,737 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,738 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,738 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:30,738 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:30,738 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:30,738 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,738 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:30,740 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,740 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,740 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:30,740 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:30,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:30,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:30,749 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:30,749 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:30,749 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:30,749 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,749 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:30,752 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,752 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,752 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:30,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:30,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:30,753 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,755 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:30,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,758 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:30,758 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:30,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:30,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,759 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:30,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:30,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:30,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:30,759 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:30,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:30,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:30,859 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:30,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:30,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:30,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:30,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,869 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:30,869 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,869 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:30,869 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:30,870 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,870 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,871 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,871 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:30,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:30,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:30,871 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:30,871 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:30,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:30,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:30,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:30,873 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:30,873 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:30,873 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:31,289 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c0736ca37a6e4eba8302dbe1de6d2283 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0736ca37a6e4eba8302dbe1de6d2283 2023-03-23 08:54:31,695 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/bab7d640667d4b55b7004637e8995b5b 2023-03-23 08:54:31,701 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/bab7d640667d4b55b7004637e8995b5b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b 2023-03-23 08:54:31,705 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b, entries=1, sequenceid=29, filesize=5.0 K 2023-03-23 08:54:31,706 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 833ms, sequenceid=29, compaction requested=true 2023-03-23 08:54:31,706 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:31,706 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:31,706 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:31,706 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:31,706 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,706 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:31,706 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:31,708 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,708 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:31,708 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,708 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:31,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:31,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:31,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ffd66f7[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:31,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:31,710 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,712 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:31,712 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,712 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:31,714 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:31,714 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,714 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:31,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:31,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:31,714 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:31,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:31,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:31,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:31,716 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:31,716 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:31,716 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:31,716 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,716 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:31,718 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,718 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:31,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,718 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:31,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:31,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:31,719 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:31,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,721 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:31,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:31,727 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,727 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,727 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:31,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:31,728 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,828 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:31,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:31,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:31,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:31,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,834 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:31,835 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:31,835 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:31,835 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:31,835 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,835 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,836 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:31,837 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:31,837 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:31,837 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:31,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,837 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:31,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,838 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,838 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:31,838 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:31,838 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:31,838 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:31,838 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:31,838 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:31,838 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:31,838 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:31,838 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:31,838 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:32,252 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036e95190e52c24a84afe78cd50b9a6c9b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036e95190e52c24a84afe78cd50b9a6c9b 2023-03-23 08:54:32,258 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/2480982d18684e4b9781004ee6bbecfd 2023-03-23 08:54:32,262 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/2480982d18684e4b9781004ee6bbecfd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd 2023-03-23 08:54:32,267 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd, entries=1, sequenceid=33, filesize=5.0 K 2023-03-23 08:54:32,267 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 429ms, sequenceid=33, compaction requested=true 2023-03-23 08:54:32,268 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:32,268 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:32,268 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:32,268 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:32,268 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,268 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:32,268 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:32,269 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,269 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:32,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:32,270 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,270 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:32,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:32,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:32,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:32,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:32,272 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@380d8b98[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:32,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:32,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,273 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,273 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:32,273 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:32,273 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:32,274 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,274 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:32,276 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:32,276 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:32,276 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,276 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:32,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:32,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:32,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:32,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:32,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:32,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:32,278 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:32,278 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:32,279 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:32,279 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:32,279 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,279 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:32,280 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,280 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:32,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,280 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:32,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:32,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:32,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:32,281 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:32,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:32,282 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:32,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,284 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,284 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:32,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:32,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:32,284 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:32,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:32,284 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:32,284 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:32,284 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:32,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:32,285 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:32,385 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:32,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:32,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:32,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:32,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,393 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:32,393 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:32,393 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:32,393 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:32,394 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,394 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,395 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,395 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:32,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:32,396 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:32,396 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:32,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:32,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:32,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:32,398 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:32,398 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:32,398 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:32,398 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:32,811 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090709653fb4595b4684a79d186ba8420e5f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090709653fb4595b4684a79d186ba8420e5f 2023-03-23 08:54:33,217 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/f2c5a3b32b70433bb9905e8cc9cb4868 2023-03-23 08:54:33,222 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/f2c5a3b32b70433bb9905e8cc9cb4868 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868 2023-03-23 08:54:33,226 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868, entries=1, sequenceid=37, filesize=5.0 K 2023-03-23 08:54:33,227 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 829ms, sequenceid=37, compaction requested=true 2023-03-23 08:54:33,228 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:33,228 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:33,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:33,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:33,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:33,228 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:33,230 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,230 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:33,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:33,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d0542cc[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:33,232 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:33,232 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,233 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:33,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:33,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:33,233 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,233 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:33,235 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:33,235 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,235 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:33,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:33,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:33,235 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:33,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:33,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:33,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:33,238 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:33,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:33,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:33,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,238 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:33,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:33,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:33,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:33,240 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:33,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:33,241 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:33,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,247 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:33,247 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:33,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:33,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-23 08:54:33,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:33,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:33,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:33,248 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:33,248 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:33,276 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:54:33,348 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:33,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:33,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:33,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:33,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,355 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:33,355 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:33,355 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-23 08:54:33,355 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-23 08:54:33,355 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,356 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,360 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,360 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:33,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:33,360 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:33,360 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:33,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:33,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-23 08:54:33,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:33,361 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-23 08:54:33,361 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:33,361 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-23 08:54:33,362 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:33,362 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:33,362 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:33,362 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. started... 2023-03-23 08:54:33,362 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6567fac53213b6f4d692c06a6cbc7ae8 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:33,775 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112039fd9523c4f54b3d9c4732b0ae50b96b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112039fd9523c4f54b3d9c4732b0ae50b96b 2023-03-23 08:54:34,180 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/732a919cbe784b50801d0ee0979b6cc0 2023-03-23 08:54:34,185 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/732a919cbe784b50801d0ee0979b6cc0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0 2023-03-23 08:54:34,188 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0, entries=1, sequenceid=41, filesize=5.0 K 2023-03-23 08:54:34,189 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 6567fac53213b6f4d692c06a6cbc7ae8 in 827ms, sequenceid=41, compaction requested=true 2023-03-23 08:54:34,189 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:34,189 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:34,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:34,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:34,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-23 08:54:34,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-23 08:54:34,191 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,191 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:34,191 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,191 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:34,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:34,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-23 08:54:34,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c67acce[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:34,192 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-23 08:54:34,192 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,194 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,194 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-23 08:54:34,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:34,194 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-23 08:54:34,194 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:34,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-23 08:54:34,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:34,196 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,196 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-23 08:54:34,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:34,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:34,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:34,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:34,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:34,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:34,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-23 08:54:34,198 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-23 08:54:34,198 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:34,198 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:34,198 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,198 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:34,200 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,200 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:34,200 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:34,200 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:34,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:34,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,201 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:34,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:34,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,203 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:34,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:34,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:34,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:34,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:34,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:34,203 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:34,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:34,204 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:34,304 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:34,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:34,306 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:34,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:34,308 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:34,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:34,309 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562274309 2023-03-23 08:54:34,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:34,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-23 08:54:34,311 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:34,311 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x2773fab0 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:34,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7feeddc7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:34,316 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,153 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031b4ce898cbe54817886b71c86ec2e18c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601031b4ce898cbe54817886b71c86ec2e18c 2023-03-23 08:54:35,154 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:35,166 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:35,168 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59212, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:35,174 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:35,175 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44488, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:35,184 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53aeedb715cb4fdd94ecf8ee9e534039 first=Optional[r3] last=Optional[r6] 2023-03-23 08:54:35,185 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53aeedb715cb4fdd94ecf8ee9e534039}] 2023-03-23 08:54:35,189 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53aeedb715cb4fdd94ecf8ee9e534039 for inclusion in 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:35,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-23 08:54:35,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:35,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:35,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53aeedb715cb4fdd94ecf8ee9e534039 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kn08hpad8ortito1mufvdjp7umfo3kl9bhmbtihbttdot6c11mk0fv5nmd63vja8/family1/53aeedb715cb4fdd94ecf8ee9e534039 2023-03-23 08:54:35,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kn08hpad8ortito1mufvdjp7umfo3kl9bhmbtihbttdot6c11mk0fv5nmd63vja8/family1/53aeedb715cb4fdd94ecf8ee9e534039 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_ 2023-03-23 08:54:35,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kn08hpad8ortito1mufvdjp7umfo3kl9bhmbtihbttdot6c11mk0fv5nmd63vja8/family1/53aeedb715cb4fdd94ecf8ee9e534039 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_ - updating store file list. 2023-03-23 08:54:35,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_ into 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:35,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kn08hpad8ortito1mufvdjp7umfo3kl9bhmbtihbttdot6c11mk0fv5nmd63vja8/family1/53aeedb715cb4fdd94ecf8ee9e534039 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_) 2023-03-23 08:54:35,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kn08hpad8ortito1mufvdjp7umfo3kl9bhmbtihbttdot6c11mk0fv5nmd63vja8/family1/53aeedb715cb4fdd94ecf8ee9e534039 2023-03-23 08:54:35,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:54:35,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:54:35,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 6567fac53213b6f4d692c06a6cbc7ae8/family1 is initiating minor compaction 2023-03-23 08:54:35,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3b88bfdd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:35,202 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6567fac53213b6f4d692c06a6cbc7ae8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:35,202 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp, totalSize=50.4 K 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting fa85e5852ce44d55a238e9498138435d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 7985e8493d3f425abc5c3c4131793812, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting d74ed6825dfc4781bd7592b86e29f015, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 48b24ab7d87d44579e1f97a8e141ae1d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f67044ebdc6d4172801e82c3d7bb86b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting bab7d640667d4b55b7004637e8995b5b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 2480982d18684e4b9781004ee6bbecfd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting f2c5a3b32b70433bb9905e8cc9cb4868, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 732a919cbe784b50801d0ee0979b6cc0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:54:35,202 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 744651982b83472eb5c6c260a8aea066_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:54:35,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:35,204 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:35,206 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c6e256d620e3492c82012d6b9a699a8c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c6e256d620e3492c82012d6b9a699a8c 2023-03-23 08:54:35,207 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512317bb2df8b09c04c309196c1f7d1534a25 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512317bb2df8b09c04c309196c1f7d1534a25 2023-03-23 08:54:35,208 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0736ca37a6e4eba8302dbe1de6d2283 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0736ca37a6e4eba8302dbe1de6d2283 2023-03-23 08:54:35,209 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036e95190e52c24a84afe78cd50b9a6c9b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601036e95190e52c24a84afe78cd50b9a6c9b 2023-03-23 08:54:35,209 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-23 08:54:35,209 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,210 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:35,226 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6567fac53213b6f4d692c06a6cbc7ae8#family1#compaction#61 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:35,647 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/5f9947fbd1bd47149ee287cc23b2e8c2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/5f9947fbd1bd47149ee287cc23b2e8c2 2023-03-23 08:54:35,652 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 6567fac53213b6f4d692c06a6cbc7ae8/family1 of 6567fac53213b6f4d692c06a6cbc7ae8 into 5f9947fbd1bd47149ee287cc23b2e8c2(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:35,652 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:35,652 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679561675201; duration=0sec 2023-03-23 08:54:35,652 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:35,731 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_] to archive 2023-03-23 08:54:35,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:35,733 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/fa85e5852ce44d55a238e9498138435d 2023-03-23 08:54:35,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/7985e8493d3f425abc5c3c4131793812 2023-03-23 08:54:35,736 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/d74ed6825dfc4781bd7592b86e29f015 2023-03-23 08:54:35,737 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/48b24ab7d87d44579e1f97a8e141ae1d 2023-03-23 08:54:35,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f67044ebdc6d4172801e82c3d7bb86b0 2023-03-23 08:54:35,739 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/bab7d640667d4b55b7004637e8995b5b 2023-03-23 08:54:35,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/2480982d18684e4b9781004ee6bbecfd 2023-03-23 08:54:35,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/f2c5a3b32b70433bb9905e8cc9cb4868 2023-03-23 08:54:35,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/732a919cbe784b50801d0ee0979b6cc0 2023-03-23 08:54:35,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/744651982b83472eb5c6c260a8aea066_SeqId_44_ 2023-03-23 08:54:36,049 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120561e3188220614055919fa5d84e12749e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120561e3188220614055919fa5d84e12749e 2023-03-23 08:54:36,051 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:36,074 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e60d793d2a0d4107adb78e50f5cee663 first=Optional[r00] last=Optional[r2] 2023-03-23 08:54:36,075 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e60d793d2a0d4107adb78e50f5cee663}] 2023-03-23 08:54:36,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e60d793d2a0d4107adb78e50f5cee663 for inclusion in 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:36,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-23 08:54:36,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:36,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:36,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/e60d793d2a0d4107adb78e50f5cee663 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__todenadmqra410v1vlbl8brl0v486dl92eb2f477ka59mm2rfnlbdd0pq9ht9p3b/family1/e60d793d2a0d4107adb78e50f5cee663 2023-03-23 08:54:36,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__todenadmqra410v1vlbl8brl0v486dl92eb2f477ka59mm2rfnlbdd0pq9ht9p3b/family1/e60d793d2a0d4107adb78e50f5cee663 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_ 2023-03-23 08:54:36,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__todenadmqra410v1vlbl8brl0v486dl92eb2f477ka59mm2rfnlbdd0pq9ht9p3b/family1/e60d793d2a0d4107adb78e50f5cee663 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_ - updating store file list. 2023-03-23 08:54:36,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_ into 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:36,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__todenadmqra410v1vlbl8brl0v486dl92eb2f477ka59mm2rfnlbdd0pq9ht9p3b/family1/e60d793d2a0d4107adb78e50f5cee663 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_) 2023-03-23 08:54:36,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__todenadmqra410v1vlbl8brl0v486dl92eb2f477ka59mm2rfnlbdd0pq9ht9p3b/family1/e60d793d2a0d4107adb78e50f5cee663 2023-03-23 08:54:36,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:36,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:36,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 6567fac53213b6f4d692c06a6cbc7ae8/family1 is initiating minor compaction (all files) 2023-03-23 08:54:36,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@719aa2f0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:36,091 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6567fac53213b6f4d692c06a6cbc7ae8/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:36,091 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/5f9947fbd1bd47149ee287cc23b2e8c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp, totalSize=390.1 K 2023-03-23 08:54:36,092 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 1172c073a7df45528aef3d8d14718473, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:54:36,092 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 5f9947fbd1bd47149ee287cc23b2e8c2, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:54:36,093 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841145 2023-03-23 08:54:36,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:36,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:36,095 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113009486e01eb7548fc89fbe87b2d07f935 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113009486e01eb7548fc89fbe87b2d07f935 2023-03-23 08:54:36,096 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014c80db130dd44f93aeea23b4c8621300 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014c80db130dd44f93aeea23b4c8621300 2023-03-23 08:54:36,098 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d7a1ff79ee89483cbac7f623be032fc0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d7a1ff79ee89483cbac7f623be032fc0 2023-03-23 08:54:36,098 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-23 08:54:36,098 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-23 08:54:36,098 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:36,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:36,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:36,099 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:36,101 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.7930 sec 2023-03-23 08:54:36,109 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6567fac53213b6f4d692c06a6cbc7ae8#family1#compaction#62 average throughput is 22.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:36,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-23 08:54:36,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:36,587 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:36,588 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:36,588 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:36,590 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-23 08:54:36,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE}] 2023-03-23 08:54:36,593 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE 2023-03-23 08:54:36,593 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:36,594 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561676593"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561676593"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561676593"}]},"ts":"1679561676593"} 2023-03-23 08:54:36,595 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:36,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:36,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6567fac53213b6f4d692c06a6cbc7ae8, disabling compactions & flushes 2023-03-23 08:54:36,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,330 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/.tmp/family1/c8d9e16535dd4d3da540a23eb26e99a1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/c8d9e16535dd4d3da540a23eb26e99a1 2023-03-23 08:54:37,335 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 6567fac53213b6f4d692c06a6cbc7ae8/family1 of 6567fac53213b6f4d692c06a6cbc7ae8 into c8d9e16535dd4d3da540a23eb26e99a1(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:54:37,335 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:37,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,335 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1679561676091; duration=1sec 2023-03-23 08:54:37,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. after waiting 0 ms 2023-03-23 08:54:37,335 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:37,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,336 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/5f9947fbd1bd47149ee287cc23b2e8c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_] to archive 2023-03-23 08:54:37,337 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:37,339 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1172c073a7df45528aef3d8d14718473 2023-03-23 08:54:37,340 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/5f9947fbd1bd47149ee287cc23b2e8c2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/5f9947fbd1bd47149ee287cc23b2e8c2 2023-03-23 08:54:37,341 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/ca5b41980a144252b9c11171bd9eaf6e_SeqId_47_ 2023-03-23 08:54:37,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-23 08:54:37,346 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:37,346 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 6567fac53213b6f4d692c06a6cbc7ae8 to self. 2023-03-23 08:54:37,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,348 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=CLOSED 2023-03-23 08:54:37,348 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561677348"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561677348"}]},"ts":"1679561677348"} 2023-03-23 08:54:37,351 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-23 08:54:37,351 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941 in 754 msec 2023-03-23 08:54:37,351 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=true 2023-03-23 08:54:37,502 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:37,502 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561677502"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561677502"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561677502"}]},"ts":"1679561677502"} 2023-03-23 08:54:37,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:37,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6567fac53213b6f4d692c06a6cbc7ae8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:37,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:37,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,660 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,661 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:37,661 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:37,661 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6567fac53213b6f4d692c06a6cbc7ae8 columnFamilyName family1 2023-03-23 08:54:37,666 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/c8d9e16535dd4d3da540a23eb26e99a1 2023-03-23 08:54:37,666 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(310): Store=6567fac53213b6f4d692c06a6cbc7ae8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:37,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:37,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6567fac53213b6f4d692c06a6cbc7ae8; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65266596, jitterRate=-0.027451932430267334}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:37,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:37,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., pid=47, masterSystemTime=1679561677655 2023-03-23 08:54:37,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:37,673 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:37,673 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561677673"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561677673"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561677673"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561677673"}]},"ts":"1679561677673"} 2023-03-23 08:54:37,676 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-23 08:54:37,676 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941 in 172 msec 2023-03-23 08:54:37,677 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-23 08:54:37,678 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE in 1.0840 sec 2023-03-23 08:54:37,680 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-23 08:54:37,680 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0880 sec 2023-03-23 08:54:37,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5030 sec 2023-03-23 08:54:37,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-23 08:54:37,683 INFO [Listener at localhost/38371] compactions.TestMobCompactor(278): alter status finished 2023-03-23 08:54:37,684 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:37,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:37,686 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:37,686 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:37,686 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562277686 2023-03-23 08:54:37,687 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:37,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-23 08:54:37,688 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:37,688 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x6dcaf95a to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:37,693 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@644301c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:37,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:37,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:37,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:37,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:37,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:38,522 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128380ed92de4aa4b7db0d1d61eb1febe44 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128380ed92de4aa4b7db0d1d61eb1febe44 2023-03-23 08:54:38,523 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:38,536 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:38,537 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59226, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:38,547 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:38,549 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44494, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:38,557 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9d785a8db01d4a02807412b2a12dbcb9 first=Optional[r05] last=Optional[r8] 2023-03-23 08:54:38,557 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=51 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9d785a8db01d4a02807412b2a12dbcb9}] 2023-03-23 08:54:38,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9d785a8db01d4a02807412b2a12dbcb9 for inclusion in 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:38,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-23 08:54:38,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:38,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:38,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9d785a8db01d4a02807412b2a12dbcb9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g27jvrfngvhpnp54mtt0bsqikaogsb7241mbdmntcjn6f1qjbp29d7955f55epqb/family1/9d785a8db01d4a02807412b2a12dbcb9 2023-03-23 08:54:38,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g27jvrfngvhpnp54mtt0bsqikaogsb7241mbdmntcjn6f1qjbp29d7955f55epqb/family1/9d785a8db01d4a02807412b2a12dbcb9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1571197650044ccfb50adc769b8efca5_SeqId_53_ 2023-03-23 08:54:38,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g27jvrfngvhpnp54mtt0bsqikaogsb7241mbdmntcjn6f1qjbp29d7955f55epqb/family1/9d785a8db01d4a02807412b2a12dbcb9 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1571197650044ccfb50adc769b8efca5_SeqId_53_ - updating store file list. 2023-03-23 08:54:38,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1571197650044ccfb50adc769b8efca5_SeqId_53_ into 6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:38,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g27jvrfngvhpnp54mtt0bsqikaogsb7241mbdmntcjn6f1qjbp29d7955f55epqb/family1/9d785a8db01d4a02807412b2a12dbcb9 into 6567fac53213b6f4d692c06a6cbc7ae8/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1571197650044ccfb50adc769b8efca5_SeqId_53_) 2023-03-23 08:54:38,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g27jvrfngvhpnp54mtt0bsqikaogsb7241mbdmntcjn6f1qjbp29d7955f55epqb/family1/9d785a8db01d4a02807412b2a12dbcb9 2023-03-23 08:54:38,575 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:38,576 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:38,578 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112039fd9523c4f54b3d9c4732b0ae50b96b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112039fd9523c4f54b3d9c4732b0ae50b96b 2023-03-23 08:54:38,579 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112802b06dd3901d4280bff8cc6467968cf2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112802b06dd3901d4280bff8cc6467968cf2 2023-03-23 08:54:38,579 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-23 08:54:38,579 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-23 08:54:38,579 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:38,579 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:38,579 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:38,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:38,579 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:38,580 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:38,582 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 896 msec 2023-03-23 08:54:38,620 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-23 08:54:38,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:54:38,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:39,041 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-23 08:54:39,042 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:39,042 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-23 08:54:39,043 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-23 08:54:39,046 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE}] 2023-03-23 08:54:39,047 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE 2023-03-23 08:54:39,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:39,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561679047"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561679047"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561679047"}]},"ts":"1679561679047"} 2023-03-23 08:54:39,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:39,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6567fac53213b6f4d692c06a6cbc7ae8, disabling compactions & flushes 2023-03-23 08:54:39,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. after waiting 0 ms 2023-03-23 08:54:39,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-23 08:54:39,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:39,208 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 6567fac53213b6f4d692c06a6cbc7ae8 to self. 2023-03-23 08:54:39,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,210 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=CLOSED 2023-03-23 08:54:39,210 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561679210"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561679210"}]},"ts":"1679561679210"} 2023-03-23 08:54:39,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-23 08:54:39,212 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941 in 162 msec 2023-03-23 08:54:39,213 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=true 2023-03-23 08:54:39,363 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:39,363 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561679363"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561679363"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561679363"}]},"ts":"1679561679363"} 2023-03-23 08:54:39,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:39,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6567fac53213b6f4d692c06a6cbc7ae8, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:39,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:39,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,521 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,521 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:39,522 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1 2023-03-23 08:54:39,522 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6567fac53213b6f4d692c06a6cbc7ae8 columnFamilyName family1 2023-03-23 08:54:39,526 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/1571197650044ccfb50adc769b8efca5_SeqId_53_ 2023-03-23 08:54:39,529 DEBUG [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/family1/c8d9e16535dd4d3da540a23eb26e99a1 2023-03-23 08:54:39,529 INFO [StoreOpener-6567fac53213b6f4d692c06a6cbc7ae8-1] regionserver.HStore(310): Store=6567fac53213b6f4d692c06a6cbc7ae8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:39,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:54:39,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6567fac53213b6f4d692c06a6cbc7ae8; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67580375, jitterRate=0.007026061415672302}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:39,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:54:39,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8., pid=53, masterSystemTime=1679561679517 2023-03-23 08:54:39,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:54:39,536 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=6567fac53213b6f4d692c06a6cbc7ae8, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:39,536 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679561679535"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561679535"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561679535"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561679535"}]},"ts":"1679561679535"} 2023-03-23 08:54:39,538 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-23 08:54:39,538 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 6567fac53213b6f4d692c06a6cbc7ae8, server=jenkins-hbase4.apache.org,32923,1679561616941 in 172 msec 2023-03-23 08:54:39,539 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-23 08:54:39,539 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=6567fac53213b6f4d692c06a6cbc7ae8, REOPEN/MOVE in 492 msec 2023-03-23 08:54:39,540 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-23 08:54:39,540 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 496 msec 2023-03-23 08:54:39,542 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 910 msec 2023-03-23 08:54:39,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-23 08:54:39,544 INFO [Listener at localhost/38371] compactions.TestMobCompactor(278): alter status finished 2023-03-23 08:54:39,545 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-23 08:54:39,546 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:39,546 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:39,547 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:39,547 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562279547 2023-03-23 08:54:39,547 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:39,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-23 08:54:39,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:39,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x7e4da549 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:39,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27cfe48, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-23 08:54:39,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-23 08:54:39,553 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:39,554 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-23 08:54:39,555 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-03-23 08:54:39,601 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=256 (was 261), OpenFileDescriptor=639 (was 633) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=398 (was 384) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5888 (was 5930) 2023-03-23 08:54:39,612 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=256, OpenFileDescriptor=639, MaxFileDescriptor=60000, SystemLoadAverage=398, ProcessCount=169, AvailableMemoryMB=5887 2023-03-23 08:54:39,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-23 08:54:39,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:39,615 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:39,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-23 08:54:39,616 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:39,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:39,617 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:39,618 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f empty. 2023-03-23 08:54:39,618 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:39,618 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-23 08:54:39,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:39,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:40,025 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:40,027 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => fadaad8dfb5b4d4096ebaa9bf3fe536f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:40,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing fadaad8dfb5b4d4096ebaa9bf3fe536f, disabling compactions & flushes 2023-03-23 08:54:40,433 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. after waiting 0 ms 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,433 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,433 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:40,435 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:40,436 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561680436"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561680436"}]},"ts":"1679561680436"} 2023-03-23 08:54:40,437 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:54:40,438 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:40,438 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561680438"}]},"ts":"1679561680438"} 2023-03-23 08:54:40,439 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-23 08:54:40,442 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, ASSIGN}] 2023-03-23 08:54:40,443 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, ASSIGN 2023-03-23 08:54:40,444 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:40,595 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:40,595 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561680595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561680595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561680595"}]},"ts":"1679561680595"} 2023-03-23 08:54:40,597 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:40,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:40,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fadaad8dfb5b4d4096ebaa9bf3fe536f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:40,751 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:40,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,753 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,754 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:40,754 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:40,755 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadaad8dfb5b4d4096ebaa9bf3fe536f columnFamilyName family1 2023-03-23 08:54:40,756 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(310): Store=fadaad8dfb5b4d4096ebaa9bf3fe536f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:40,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:40,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fadaad8dfb5b4d4096ebaa9bf3fe536f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74399833, jitterRate=0.10864390432834625}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:40,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:40,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., pid=57, masterSystemTime=1679561680748 2023-03-23 08:54:40,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:40,784 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:40,785 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561680784"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561680784"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561680784"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561680784"}]},"ts":"1679561680784"} 2023-03-23 08:54:40,793 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-23 08:54:40,793 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 189 msec 2023-03-23 08:54:40,795 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-23 08:54:40,795 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, ASSIGN in 351 msec 2023-03-23 08:54:40,795 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:40,795 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561680795"}]},"ts":"1679561680795"} 2023-03-23 08:54:40,799 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-23 08:54:40,801 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:40,802 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1870 sec 2023-03-23 08:54:41,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-23 08:54:41,721 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-23 08:54:41,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:41,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,743 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:41,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:41,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:41,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:41,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:41,745 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:41,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:41,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:41,746 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:41,746 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:41,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:41,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:41,749 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:41,749 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:41,749 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:41,750 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:54:42,170 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511301bbef96b850e42139cc894eeb01ea6f3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511301bbef96b850e42139cc894eeb01ea6f3 2023-03-23 08:54:42,573 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/c10482abef2542b0a94dc91ffff54546 2023-03-23 08:54:42,578 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/c10482abef2542b0a94dc91ffff54546 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546 2023-03-23 08:54:42,581 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-23 08:54:42,582 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 832ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:42,582 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:42,583 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:42,583 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:42,583 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:42,584 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,584 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:42,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:42,584 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,584 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:42,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:42,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5829b36b[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:42,586 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:42,586 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,587 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:42,587 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:42,587 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,587 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:42,588 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,588 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,588 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:42,588 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:42,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:42,590 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:42,590 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:42,590 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:42,590 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,590 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:42,593 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:42,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:42,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:42,593 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,594 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:42,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:42,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:42,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:42,599 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:42,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,599 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:42,600 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:42,700 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:42,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:42,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:42,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:42,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,706 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:42,706 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:42,706 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:42,706 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:42,707 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,707 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,708 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,708 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:42,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:42,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:42,708 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:42,708 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:42,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:42,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,709 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,709 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:42,709 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:42,709 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:42,709 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:42,709 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:42,710 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:42,710 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:42,710 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:42,710 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:42,710 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:54:42,847 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:54:43,123 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128422efc3676a241aa94fc7bb785e76f36 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128422efc3676a241aa94fc7bb785e76f36 2023-03-23 08:54:43,538 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/c35ed8251f38483e977af6079b9c7db9 2023-03-23 08:54:43,543 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/c35ed8251f38483e977af6079b9c7db9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9 2023-03-23 08:54:43,547 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:54:43,548 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 838ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:43,548 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:43,548 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:43,548 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:43,554 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:43,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:43,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:43,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:43,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:43,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cacb2bd[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:43,556 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:43,556 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,557 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:43,557 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:43,557 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,557 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:43,558 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,558 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:43,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:43,561 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:43,561 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:43,561 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:43,561 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,561 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:43,563 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,563 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,563 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:43,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:43,563 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:43,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:43,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:43,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:43,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:43,568 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:43,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:43,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:43,569 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:43,669 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:43,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:43,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:43,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:43,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,678 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:43,678 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:43,678 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:43,678 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:43,678 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,678 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,680 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,680 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:43,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:43,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:43,680 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:43,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,681 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:43,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:43,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,682 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:43,682 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:43,682 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:43,682 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:43,682 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:43,684 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:43,684 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:43,684 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:43,684 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:43,684 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:44,131 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120187711497ab8048c3b201bb178fa7f884 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120187711497ab8048c3b201bb178fa7f884 2023-03-23 08:54:44,546 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/e3f68f84c9a544138f0a52820895afba 2023-03-23 08:54:44,552 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/e3f68f84c9a544138f0a52820895afba as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba 2023-03-23 08:54:44,560 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba, entries=1, sequenceid=13, filesize=5.0 K 2023-03-23 08:54:44,561 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 877ms, sequenceid=13, compaction requested=true 2023-03-23 08:54:44,561 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:44,561 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:44,561 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:44,561 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:44,561 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,561 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:44,561 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:44,563 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:44,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b3882c6[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:44,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:44,565 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,567 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,567 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:44,567 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:44,567 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:44,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:44,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:44,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:44,569 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,569 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:44,572 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:44,572 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:44,572 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:44,572 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:44,572 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,572 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:44,574 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,574 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,574 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:44,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:44,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:44,575 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,576 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:44,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:44,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:44,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:44,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:44,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:44,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:44,579 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:44,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:44,579 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:44,680 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:44,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:44,684 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:44,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:44,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,693 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:44,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:44,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:44,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:44,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,695 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:44,695 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:44,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:44,696 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:44,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,696 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:44,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:44,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:44,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:44,697 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:44,697 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:44,697 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:44,698 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:44,698 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:44,698 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:44,698 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:44,698 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:45,113 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d2d28b1e074d4a84975eba12111374f7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205d2d28b1e074d4a84975eba12111374f7 2023-03-23 08:54:45,119 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/d464f66a1616400f87e2d8ecf0c9fb28 2023-03-23 08:54:45,124 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/d464f66a1616400f87e2d8ecf0c9fb28 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28 2023-03-23 08:54:45,128 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28, entries=1, sequenceid=17, filesize=5.0 K 2023-03-23 08:54:45,129 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 431ms, sequenceid=17, compaction requested=true 2023-03-23 08:54:45,129 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:45,129 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:45,139 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,139 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:45,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:45,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a584eb3[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:45,141 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:45,141 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,143 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,144 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:45,144 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:45,144 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:45,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,144 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:45,148 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:45,148 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:45,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:45,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:45,151 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:45,151 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:45,151 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:45,151 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:45,151 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,151 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:45,154 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,154 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:45,154 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:45,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:45,155 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,157 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,161 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:45,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:45,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:45,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:45,162 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:45,162 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:45,164 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:45,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:45,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:45,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:45,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:45,262 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:45,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:45,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:45,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:45,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,268 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:45,269 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:45,269 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:45,269 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:45,269 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,269 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,272 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:45,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:45,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:45,272 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,272 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,273 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,273 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:45,274 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:45,274 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:45,274 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:45,274 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:45,274 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:45,274 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:45,274 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:45,274 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:45,317 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-23 08:54:45,688 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f6ba3668ecda4cceba2b1f6bff473270 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6ba3668ecda4cceba2b1f6bff473270 2023-03-23 08:54:46,094 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/1f88181dea9d41cd96bf83f86780135e 2023-03-23 08:54:46,100 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/1f88181dea9d41cd96bf83f86780135e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e 2023-03-23 08:54:46,105 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e, entries=1, sequenceid=21, filesize=5.0 K 2023-03-23 08:54:46,107 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 833ms, sequenceid=21, compaction requested=true 2023-03-23 08:54:46,107 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:46,107 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:46,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:46,109 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:46,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:46,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:46,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:46,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:46,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:46,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f7f0a78[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:46,111 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:46,111 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,113 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:46,113 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,113 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:46,113 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:46,116 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:46,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:46,116 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:46,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:46,119 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:46,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:46,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:46,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,119 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:46,120 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,120 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:46,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,120 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:46,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:46,121 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,122 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,125 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:46,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:46,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:46,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:46,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:46,125 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:46,126 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:46,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:46,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:46,226 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:46,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:46,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:46,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:46,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,234 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:46,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:46,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:46,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:46,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,234 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,236 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:46,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:46,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:46,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:46,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:46,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:46,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:46,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:46,238 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:46,238 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:46,238 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:46,239 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:46,655 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231677467b5eaa143679ba1aa6ae0de1fd1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231677467b5eaa143679ba1aa6ae0de1fd1 2023-03-23 08:54:47,068 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/84b10246c1e94d6cb9fade22b4cf8a34 2023-03-23 08:54:47,075 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/84b10246c1e94d6cb9fade22b4cf8a34 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34 2023-03-23 08:54:47,080 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34, entries=1, sequenceid=25, filesize=5.0 K 2023-03-23 08:54:47,081 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 842ms, sequenceid=25, compaction requested=true 2023-03-23 08:54:47,082 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:47,082 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,084 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:47,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@162f6670[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:47,086 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:47,086 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,088 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:47,088 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:47,088 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:47,088 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,088 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:47,089 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:47,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:47,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:47,092 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:47,092 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:47,092 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:47,092 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,092 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:47,095 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,095 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,096 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,108 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,109 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,115 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,115 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:47,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,116 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:47,116 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:47,116 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:47,216 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:47,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:47,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:47,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:47,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,223 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:47,223 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:47,224 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,224 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,227 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,228 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,228 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,228 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:47,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:47,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:47,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,229 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:47,229 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,229 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:47,229 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:47,229 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:47,229 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,229 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:47,229 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:47,249 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101849d3dfac359400aa0d1bc7752b35dc9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101849d3dfac359400aa0d1bc7752b35dc9 2023-03-23 08:54:47,266 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/07bb15083c7c40a987a637f2107a3eb4 2023-03-23 08:54:47,272 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/07bb15083c7c40a987a637f2107a3eb4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4 2023-03-23 08:54:47,280 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4, entries=1, sequenceid=29, filesize=5.0 K 2023-03-23 08:54:47,281 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 52ms, sequenceid=29, compaction requested=true 2023-03-23 08:54:47,281 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:47,281 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:47,283 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,283 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,283 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,283 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:47,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:47,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@594ef8c1[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:47,286 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:47,286 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,288 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:47,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:47,288 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:47,289 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,289 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:47,291 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:47,291 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,291 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,291 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:47,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:47,295 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:47,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:47,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:47,295 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,295 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:47,297 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,297 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,299 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,300 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,302 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,306 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,307 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,307 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:47,308 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,308 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:47,308 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:47,308 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:47,408 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:47,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:47,414 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:47,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:47,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,416 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:47,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:47,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,418 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,418 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,419 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,419 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:47,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:47,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,420 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,420 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:47,420 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,420 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:47,420 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,420 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:47,421 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:47,421 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:47,421 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,421 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:47,421 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:47,438 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034497b0c5c6a2442bbb03c7b6bbed4c12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034497b0c5c6a2442bbb03c7b6bbed4c12 2023-03-23 08:54:47,444 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/0e129d2e395b4947830a108afd297a54 2023-03-23 08:54:47,449 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/0e129d2e395b4947830a108afd297a54 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54 2023-03-23 08:54:47,458 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54, entries=1, sequenceid=33, filesize=5.0 K 2023-03-23 08:54:47,459 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 38ms, sequenceid=33, compaction requested=true 2023-03-23 08:54:47,459 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:47,459 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:47,459 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:47,462 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,462 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:47,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:47,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@714b088d[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:47,464 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:47,464 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,466 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:47,466 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:47,467 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,467 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:47,468 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,468 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:47,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:47,471 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:47,471 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:47,471 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:47,471 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,471 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:47,472 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:47,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:47,473 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,475 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,475 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,475 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:47,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:47,479 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:47,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:47,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:47,479 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:47,580 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:47,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:47,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:47,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:47,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,588 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:47,588 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,588 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:47,588 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:47,588 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,588 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,589 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,589 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:47,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:47,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:47,590 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:47,590 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:47,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:47,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:47,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:47,592 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:47,592 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:47,592 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:47,592 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:47,592 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:47,592 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:47,592 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:47,593 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:48,019 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907207fa372adf44acaac5cd99a94656bbe to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907207fa372adf44acaac5cd99a94656bbe 2023-03-23 08:54:48,025 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/e13742458be74278bfc68da34eb73ee4 2023-03-23 08:54:48,030 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/e13742458be74278bfc68da34eb73ee4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4 2023-03-23 08:54:48,035 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4, entries=1, sequenceid=37, filesize=5.0 K 2023-03-23 08:54:48,035 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 442ms, sequenceid=37, compaction requested=true 2023-03-23 08:54:48,036 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:48,036 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:48,036 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:48,036 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:48,036 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,036 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:48,036 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:48,038 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:48,038 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:48,039 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,039 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:48,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:48,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76e0e016[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:48,040 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:48,040 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,043 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,043 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:48,043 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:48,043 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,043 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:48,043 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:48,045 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:48,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:48,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:48,045 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:48,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:48,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:48,048 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:48,048 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:48,048 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:48,048 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,048 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:48,050 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,050 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:48,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:48,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,050 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:48,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:48,051 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:48,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,052 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:48,055 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:48,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:48,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:48,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:48,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,055 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,056 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,056 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,056 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:48,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:48,056 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:48,056 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:48,156 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:48,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:48,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:48,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-23 08:54:48,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,167 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:48,167 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:48,167 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-23 08:54:48,167 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-23 08:54:48,168 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,168 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,170 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:48,170 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:48,171 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:48,171 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-23 08:54:48,172 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:48,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-23 08:54:48,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:48,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-23 08:54:48,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:54:48,172 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:48,172 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:48,172 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. started... 2023-03-23 08:54:48,172 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fadaad8dfb5b4d4096ebaa9bf3fe536f 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:54:48,595 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112044821f33ea2d46f099f325a4ac1b172c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112044821f33ea2d46f099f325a4ac1b172c 2023-03-23 08:54:49,034 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/4b5fc8632e2d491cad79cd9b57c39f94 2023-03-23 08:54:49,041 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/4b5fc8632e2d491cad79cd9b57c39f94 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94 2023-03-23 08:54:49,046 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94, entries=1, sequenceid=41, filesize=5.0 K 2023-03-23 08:54:49,047 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fadaad8dfb5b4d4096ebaa9bf3fe536f in 875ms, sequenceid=41, compaction requested=true 2023-03-23 08:54:49,047 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:49,047 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:54:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:54:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-23 08:54:49,047 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-23 08:54:49,049 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,049 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:49,050 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,050 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:54:49,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:49,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-23 08:54:49,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b91045c[Count = 0] remaining members to acquire global barrier 2023-03-23 08:54:49,052 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-23 08:54:49,052 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,053 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-23 08:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-23 08:54:49,053 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,053 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:54:49,053 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:54:49,057 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-23 08:54:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:49,057 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-23 08:54:49,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:54:49,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-23 08:54:49,065 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-23 08:54:49,065 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:54:49,065 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:54:49,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,066 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:54:49,068 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,068 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:49,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,069 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:49,069 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:54:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:54:49,075 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:54:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-23 08:54:49,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:54:49,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:49,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:49,077 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:54:49,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-23 08:54:49,078 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:49,078 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,078 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-23 08:54:49,078 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:54:49,078 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:54:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:54:49,178 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:54:49,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:54:49,180 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:49,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:49,183 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:49,184 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:49,184 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562289184 2023-03-23 08:54:49,185 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:49,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-23 08:54:49,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:49,187 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x4a2265bd to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:49,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@335448d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:49,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,696 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103e55846953df74e3aa11f12e2d47b73c2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103e55846953df74e3aa11f12e2d47b73c2 2023-03-23 08:54:49,697 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:49,718 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:49,720 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56154, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:49,731 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:49,733 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55260, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:49,744 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b53c68acc84644a3942a5c54455a1b9a first=Optional[r3] last=Optional[r6] 2023-03-23 08:54:49,745 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b53c68acc84644a3942a5c54455a1b9a}] 2023-03-23 08:54:49,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b53c68acc84644a3942a5c54455a1b9a for inclusion in fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:49,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-23 08:54:49,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:49,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:49,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/b53c68acc84644a3942a5c54455a1b9a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s003raaqdd2lne3v8gn53348f918chdbj9sh4v8vhp1dn50ehjcvtr06btdeoq78/family1/b53c68acc84644a3942a5c54455a1b9a 2023-03-23 08:54:49,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s003raaqdd2lne3v8gn53348f918chdbj9sh4v8vhp1dn50ehjcvtr06btdeoq78/family1/b53c68acc84644a3942a5c54455a1b9a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_ 2023-03-23 08:54:49,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s003raaqdd2lne3v8gn53348f918chdbj9sh4v8vhp1dn50ehjcvtr06btdeoq78/family1/b53c68acc84644a3942a5c54455a1b9a into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_ - updating store file list. 2023-03-23 08:54:49,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_ into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:49,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s003raaqdd2lne3v8gn53348f918chdbj9sh4v8vhp1dn50ehjcvtr06btdeoq78/family1/b53c68acc84644a3942a5c54455a1b9a into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_) 2023-03-23 08:54:49,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s003raaqdd2lne3v8gn53348f918chdbj9sh4v8vhp1dn50ehjcvtr06btdeoq78/family1/b53c68acc84644a3942a5c54455a1b9a 2023-03-23 08:54:49,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:54:49,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:54:49,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 is initiating minor compaction 2023-03-23 08:54:49,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@50fc2ce9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:49,766 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:49,767 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp, totalSize=50.3 K 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting c35ed8251f38483e977af6079b9c7db9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting e3f68f84c9a544138f0a52820895afba, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting d464f66a1616400f87e2d8ecf0c9fb28, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 1f88181dea9d41cd96bf83f86780135e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 84b10246c1e94d6cb9fade22b4cf8a34, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 07bb15083c7c40a987a637f2107a3eb4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 0e129d2e395b4947830a108afd297a54, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting e13742458be74278bfc68da34eb73ee4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 4b5fc8632e2d491cad79cd9b57c39f94, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:54:49,767 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting c2be581e09fe44c598f587d9cc1c7183_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:54:49,776 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:49,777 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:49,780 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6ba3668ecda4cceba2b1f6bff473270 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6ba3668ecda4cceba2b1f6bff473270 2023-03-23 08:54:49,782 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231677467b5eaa143679ba1aa6ae0de1fd1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231677467b5eaa143679ba1aa6ae0de1fd1 2023-03-23 08:54:49,784 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101849d3dfac359400aa0d1bc7752b35dc9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101849d3dfac359400aa0d1bc7752b35dc9 2023-03-23 08:54:49,786 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034497b0c5c6a2442bbb03c7b6bbed4c12 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601034497b0c5c6a2442bbb03c7b6bbed4c12 2023-03-23 08:54:49,786 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-23 08:54:49,786 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-23 08:54:49,786 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,786 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,787 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:49,829 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): fadaad8dfb5b4d4096ebaa9bf3fe536f#family1#compaction#73 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:49,947 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:49,947 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f58e53d to 127.0.0.1:50031 2023-03-23 08:54:49,948 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:49,948 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:49,948 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x782e5dab to 127.0.0.1:50031 2023-03-23 08:54:49,948 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:49,950 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:49,950 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2773fab0 to 127.0.0.1:50031 2023-03-23 08:54:49,951 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:49,951 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:54:49,951 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6dcaf95a to 127.0.0.1:50031 2023-03-23 08:54:49,951 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:49,951 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7e4da549 to 127.0.0.1:50031 2023-03-23 08:54:49,951 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:54:49,991 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120587c9bef7c3ad47db81d49be8161cfb43 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120587c9bef7c3ad47db81d49be8161cfb43 2023-03-23 08:54:49,992 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:49,993 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/1908b3a803714681a3c86ae6fc0eb4e8 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1908b3a803714681a3c86ae6fc0eb4e8 2023-03-23 08:54:49,999 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 of fadaad8dfb5b4d4096ebaa9bf3fe536f into 1908b3a803714681a3c86ae6fc0eb4e8(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:49,999 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:49,999 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679561689766; duration=0sec 2023-03-23 08:54:49,999 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:50,039 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_] to archive 2023-03-23 08:54:50,044 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/41d35089e5eb4f3a9564f9252182cd8f first=Optional[r00] last=Optional[r2] 2023-03-23 08:54:50,044 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:50,044 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/41d35089e5eb4f3a9564f9252182cd8f}] 2023-03-23 08:54:50,046 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c35ed8251f38483e977af6079b9c7db9 2023-03-23 08:54:50,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/41d35089e5eb4f3a9564f9252182cd8f for inclusion in fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:50,048 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e3f68f84c9a544138f0a52820895afba 2023-03-23 08:54:50,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/d464f66a1616400f87e2d8ecf0c9fb28 2023-03-23 08:54:50,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-23 08:54:50,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:50,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:50,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1f88181dea9d41cd96bf83f86780135e 2023-03-23 08:54:50,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/41d35089e5eb4f3a9564f9252182cd8f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qlsi3q7cnc61msdjqsp8qpo5dbp468nmbo6isbuj5l21svjjvfa8fnta9mofs4g7/family1/41d35089e5eb4f3a9564f9252182cd8f 2023-03-23 08:54:50,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/84b10246c1e94d6cb9fade22b4cf8a34 2023-03-23 08:54:50,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/07bb15083c7c40a987a637f2107a3eb4 2023-03-23 08:54:50,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qlsi3q7cnc61msdjqsp8qpo5dbp468nmbo6isbuj5l21svjjvfa8fnta9mofs4g7/family1/41d35089e5eb4f3a9564f9252182cd8f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_ 2023-03-23 08:54:50,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qlsi3q7cnc61msdjqsp8qpo5dbp468nmbo6isbuj5l21svjjvfa8fnta9mofs4g7/family1/41d35089e5eb4f3a9564f9252182cd8f into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_ - updating store file list. 2023-03-23 08:54:50,058 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/0e129d2e395b4947830a108afd297a54 2023-03-23 08:54:50,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/e13742458be74278bfc68da34eb73ee4 2023-03-23 08:54:50,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_ into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:50,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qlsi3q7cnc61msdjqsp8qpo5dbp468nmbo6isbuj5l21svjjvfa8fnta9mofs4g7/family1/41d35089e5eb4f3a9564f9252182cd8f into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_) 2023-03-23 08:54:50,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/4b5fc8632e2d491cad79cd9b57c39f94 2023-03-23 08:54:50,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__qlsi3q7cnc61msdjqsp8qpo5dbp468nmbo6isbuj5l21svjjvfa8fnta9mofs4g7/family1/41d35089e5eb4f3a9564f9252182cd8f 2023-03-23 08:54:50,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:54:50,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:54:50,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 is initiating minor compaction (all files) 2023-03-23 08:54:50,064 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c2be581e09fe44c598f587d9cc1c7183_SeqId_44_ 2023-03-23 08:54:50,064 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1908b3a803714681a3c86ae6fc0eb4e8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp, totalSize=388.1 K 2023-03-23 08:54:50,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@60d2d7b9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:50,065 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting c10482abef2542b0a94dc91ffff54546, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:54:50,065 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 1908b3a803714681a3c86ae6fc0eb4e8, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:54:50,066 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting 74f4f2507ab746408c5a5a6593000c74_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878841145 2023-03-23 08:54:50,066 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:50,067 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:50,069 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511301bbef96b850e42139cc894eeb01ea6f3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511301bbef96b850e42139cc894eeb01ea6f3 2023-03-23 08:54:50,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120187711497ab8048c3b201bb178fa7f884 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120187711497ab8048c3b201bb178fa7f884 2023-03-23 08:54:50,072 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205d2d28b1e074d4a84975eba12111374f7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205d2d28b1e074d4a84975eba12111374f7 2023-03-23 08:54:50,072 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-23 08:54:50,072 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-23 08:54:50,072 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:50,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:50,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:50,074 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:50,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 893 msec 2023-03-23 08:54:50,087 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): fadaad8dfb5b4d4096ebaa9bf3fe536f#family1#compaction#74 average throughput is 14.97 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:50,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-23 08:54:50,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:50,574 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:50,575 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:50,575 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:50,577 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-23 08:54:50,580 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE}] 2023-03-23 08:54:50,580 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE 2023-03-23 08:54:50,581 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:50,581 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561690581"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561690581"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561690581"}]},"ts":"1679561690581"} 2023-03-23 08:54:50,582 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:50,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:50,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fadaad8dfb5b4d4096ebaa9bf3fe536f, disabling compactions & flushes 2023-03-23 08:54:50,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,928 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/.tmp/family1/547467adf01c49c98af1165ca5122b03 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/547467adf01c49c98af1165ca5122b03 2023-03-23 08:54:50,933 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 of fadaad8dfb5b4d4096ebaa9bf3fe536f into 547467adf01c49c98af1165ca5122b03(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:50,933 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:50,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. after waiting 0 ms 2023-03-23 08:54:50,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,933 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1679561690064; duration=0sec 2023-03-23 08:54:50,933 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:50,937 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1908b3a803714681a3c86ae6fc0eb4e8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_] to archive 2023-03-23 08:54:50,938 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:50,941 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/c10482abef2542b0a94dc91ffff54546 2023-03-23 08:54:50,942 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1908b3a803714681a3c86ae6fc0eb4e8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/1908b3a803714681a3c86ae6fc0eb4e8 2023-03-23 08:54:50,944 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/74f4f2507ab746408c5a5a6593000c74_SeqId_47_ 2023-03-23 08:54:50,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-23 08:54:50,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:50,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:50,949 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: fadaad8dfb5b4d4096ebaa9bf3fe536f to self. 2023-03-23 08:54:50,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:50,951 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=CLOSED 2023-03-23 08:54:50,951 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561690951"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561690951"}]},"ts":"1679561690951"} 2023-03-23 08:54:50,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-23 08:54:50,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 370 msec 2023-03-23 08:54:50,954 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=true 2023-03-23 08:54:51,104 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:51,104 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561691104"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561691104"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561691104"}]},"ts":"1679561691104"} 2023-03-23 08:54:51,106 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:51,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:51,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fadaad8dfb5b4d4096ebaa9bf3fe536f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:51,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:51,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,262 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,263 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:51,264 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:51,264 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadaad8dfb5b4d4096ebaa9bf3fe536f columnFamilyName family1 2023-03-23 08:54:51,270 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/547467adf01c49c98af1165ca5122b03 2023-03-23 08:54:51,270 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(310): Store=fadaad8dfb5b4d4096ebaa9bf3fe536f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:51,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:51,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fadaad8dfb5b4d4096ebaa9bf3fe536f; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61531033, jitterRate=-0.08311615884304047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:51,276 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:51,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., pid=63, masterSystemTime=1679561691258 2023-03-23 08:54:51,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:51,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:51,278 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:51,278 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561691278"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561691278"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561691278"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561691278"}]},"ts":"1679561691278"} 2023-03-23 08:54:51,281 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-23 08:54:51,281 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 174 msec 2023-03-23 08:54:51,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-23 08:54:51,283 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE in 701 msec 2023-03-23 08:54:51,285 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-23 08:54:51,285 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 707 msec 2023-03-23 08:54:51,288 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1260 sec 2023-03-23 08:54:51,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-23 08:54:51,290 INFO [Listener at localhost/38371] compactions.TestMobCompactor(278): alter status finished 2023-03-23 08:54:51,291 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:51,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:51,297 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:51,297 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562291297 2023-03-23 08:54:51,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:51,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:51,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-23 08:54:51,299 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:51,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x5d28c898 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:51,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1774aef4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:51,329 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:51,329 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:51,329 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:51,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:51,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,211 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511285c433a628d644670b7120a59493c2df5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511285c433a628d644670b7120a59493c2df5 2023-03-23 08:54:52,212 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:52,225 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:52,226 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56170, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:52,232 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:52,234 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55262, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:52,249 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86a361a5018d4e1db88f80bfd9965310 first=Optional[r05] last=Optional[r8] 2023-03-23 08:54:52,250 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=51 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86a361a5018d4e1db88f80bfd9965310}] 2023-03-23 08:54:52,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86a361a5018d4e1db88f80bfd9965310 for inclusion in fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:52,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-23 08:54:52,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:52,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:52,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/86a361a5018d4e1db88f80bfd9965310 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ek2441mgpn29jqqimqsm9t3plr73fgk7rgggal12027mv09j66imr2sonhfbokfv/family1/86a361a5018d4e1db88f80bfd9965310 2023-03-23 08:54:52,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ek2441mgpn29jqqimqsm9t3plr73fgk7rgggal12027mv09j66imr2sonhfbokfv/family1/86a361a5018d4e1db88f80bfd9965310 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/9209a37045684dc8ba1e1452766859b5_SeqId_53_ 2023-03-23 08:54:52,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ek2441mgpn29jqqimqsm9t3plr73fgk7rgggal12027mv09j66imr2sonhfbokfv/family1/86a361a5018d4e1db88f80bfd9965310 into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/9209a37045684dc8ba1e1452766859b5_SeqId_53_ - updating store file list. 2023-03-23 08:54:52,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/9209a37045684dc8ba1e1452766859b5_SeqId_53_ into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:52,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ek2441mgpn29jqqimqsm9t3plr73fgk7rgggal12027mv09j66imr2sonhfbokfv/family1/86a361a5018d4e1db88f80bfd9965310 into fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/9209a37045684dc8ba1e1452766859b5_SeqId_53_) 2023-03-23 08:54:52,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__ek2441mgpn29jqqimqsm9t3plr73fgk7rgggal12027mv09j66imr2sonhfbokfv/family1/86a361a5018d4e1db88f80bfd9965310 2023-03-23 08:54:52,277 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:52,278 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:52,280 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112044821f33ea2d46f099f325a4ac1b172c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112044821f33ea2d46f099f325a4ac1b172c 2023-03-23 08:54:52,281 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128422efc3676a241aa94fc7bb785e76f36 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128422efc3676a241aa94fc7bb785e76f36 2023-03-23 08:54:52,281 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-23 08:54:52,281 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-23 08:54:52,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:52,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:52,283 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:52,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 991 msec 2023-03-23 08:54:52,355 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-23 08:54:52,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:52,374 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-23 08:54:52,375 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-23 08:54:52,375 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-23 08:54:52,377 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-23 08:54:52,380 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE}] 2023-03-23 08:54:52,380 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE 2023-03-23 08:54:52,381 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:52,381 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561692381"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561692381"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561692381"}]},"ts":"1679561692381"} 2023-03-23 08:54:52,382 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:52,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fadaad8dfb5b4d4096ebaa9bf3fe536f, disabling compactions & flushes 2023-03-23 08:54:52,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. after waiting 0 ms 2023-03-23 08:54:52,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-23 08:54:52,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:52,545 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: fadaad8dfb5b4d4096ebaa9bf3fe536f to self. 2023-03-23 08:54:52,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=CLOSED 2023-03-23 08:54:52,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561692547"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561692547"}]},"ts":"1679561692547"} 2023-03-23 08:54:52,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-23 08:54:52,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 167 msec 2023-03-23 08:54:52,551 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=true 2023-03-23 08:54:52,702 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:52,702 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561692702"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561692702"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561692702"}]},"ts":"1679561692702"} 2023-03-23 08:54:52,704 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:52,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fadaad8dfb5b4d4096ebaa9bf3fe536f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:52,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:52,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,860 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,861 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:52,861 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1 2023-03-23 08:54:52,862 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadaad8dfb5b4d4096ebaa9bf3fe536f columnFamilyName family1 2023-03-23 08:54:52,867 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/547467adf01c49c98af1165ca5122b03 2023-03-23 08:54:52,871 DEBUG [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/family1/9209a37045684dc8ba1e1452766859b5_SeqId_53_ 2023-03-23 08:54:52,871 INFO [StoreOpener-fadaad8dfb5b4d4096ebaa9bf3fe536f-1] regionserver.HStore(310): Store=fadaad8dfb5b4d4096ebaa9bf3fe536f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:52,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:54:52,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fadaad8dfb5b4d4096ebaa9bf3fe536f; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66263931, jitterRate=-0.012590482831001282}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:52,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:54:52,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f., pid=69, masterSystemTime=1679561692856 2023-03-23 08:54:52,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:54:52,880 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=fadaad8dfb5b4d4096ebaa9bf3fe536f, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:52,880 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679561692880"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561692880"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561692880"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561692880"}]},"ts":"1679561692880"} 2023-03-23 08:54:52,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-23 08:54:52,883 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure fadaad8dfb5b4d4096ebaa9bf3fe536f, server=jenkins-hbase4.apache.org,32923,1679561616941 in 177 msec 2023-03-23 08:54:52,885 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-23 08:54:52,885 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=fadaad8dfb5b4d4096ebaa9bf3fe536f, REOPEN/MOVE in 503 msec 2023-03-23 08:54:52,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-23 08:54:52,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 508 msec 2023-03-23 08:54:52,889 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 532 msec 2023-03-23 08:54:52,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-23 08:54:52,891 INFO [Listener at localhost/38371] compactions.TestMobCompactor(278): alter status finished 2023-03-23 08:54:52,892 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-23 08:54:52,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:52,894 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:52,895 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562292895 2023-03-23 08:54:52,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:52,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:54:52,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-23 08:54:52,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:52,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x56f46416 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:52,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7ccfa66b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:52,908 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-23 08:54:52,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-23 08:54:52,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:52,910 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-23 08:54:52,911 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 17 msec 2023-03-23 08:54:52,965 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=246 (was 256), OpenFileDescriptor=629 (was 639), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 398) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5765 (was 5887) 2023-03-23 08:54:52,980 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=246, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=413, ProcessCount=169, AvailableMemoryMB=5764 2023-03-23 08:54:52,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-23 08:54:52,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:52,987 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:52,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-23 08:54:52,987 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:52,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:52,989 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:52,990 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 empty. 2023-03-23 08:54:52,990 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:52,990 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-23 08:54:53,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:53,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:53,407 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:53,409 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => f36cd72431a8d4b98b336297a112e839, NAME => 'testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:53,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:53,815 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:53,816 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing f36cd72431a8d4b98b336297a112e839, disabling compactions & flushes 2023-03-23 08:54:53,816 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:53,816 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:53,816 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. after waiting 0 ms 2023-03-23 08:54:53,816 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:53,816 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:53,816 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:53,818 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:53,819 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679561693819"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561693819"}]},"ts":"1679561693819"} 2023-03-23 08:54:53,820 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:54:53,821 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:53,821 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561693821"}]},"ts":"1679561693821"} 2023-03-23 08:54:53,822 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-23 08:54:53,826 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, ASSIGN}] 2023-03-23 08:54:53,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, ASSIGN 2023-03-23 08:54:53,828 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:53,979 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f36cd72431a8d4b98b336297a112e839, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:53,980 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679561693979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561693979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561693979"}]},"ts":"1679561693979"} 2023-03-23 08:54:53,981 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure f36cd72431a8d4b98b336297a112e839, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:54,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:54,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:54,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f36cd72431a8d4b98b336297a112e839, NAME => 'testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:54:54,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-23 08:54:54,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-23 08:54:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,141 INFO [StoreOpener-f36cd72431a8d4b98b336297a112e839-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,142 DEBUG [StoreOpener-f36cd72431a8d4b98b336297a112e839-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1 2023-03-23 08:54:54,142 DEBUG [StoreOpener-f36cd72431a8d4b98b336297a112e839-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1 2023-03-23 08:54:54,143 INFO [StoreOpener-f36cd72431a8d4b98b336297a112e839-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f36cd72431a8d4b98b336297a112e839 columnFamilyName f1 2023-03-23 08:54:54,144 INFO [StoreOpener-f36cd72431a8d4b98b336297a112e839-1] regionserver.HStore(310): Store=f36cd72431a8d4b98b336297a112e839/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:54,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:54,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:54,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f36cd72431a8d4b98b336297a112e839; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64721019, jitterRate=-0.03558166325092316}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:54:54,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:54,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839., pid=73, masterSystemTime=1679561694134 2023-03-23 08:54:54,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:54,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:54,153 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f36cd72431a8d4b98b336297a112e839, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:54,153 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679561694153"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561694153"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561694153"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561694153"}]},"ts":"1679561694153"} 2023-03-23 08:54:54,156 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-23 08:54:54,156 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure f36cd72431a8d4b98b336297a112e839, server=jenkins-hbase4.apache.org,32923,1679561616941 in 174 msec 2023-03-23 08:54:54,158 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-23 08:54:54,158 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, ASSIGN in 330 msec 2023-03-23 08:54:54,159 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:54,159 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561694159"}]},"ts":"1679561694159"} 2023-03-23 08:54:54,160 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-23 08:54:54,166 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:54,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1820 sec 2023-03-23 08:54:55,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-23 08:54:55,094 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-23 08:54:55,094 DEBUG [Listener at localhost/38371] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-23 08:54:55,097 INFO [Listener at localhost/38371] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-23 08:54:55,101 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-23 08:54:55,101 INFO [Listener at localhost/38371] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-23 08:54:55,101 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-23 08:54:55,106 INFO [Listener at localhost/38371] regionserver.HRegion(2745): Flushing f36cd72431a8d4b98b336297a112e839 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-23 08:54:55,128 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e4e8d059bf07495aacd9f5f0057823fc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230323e4e8d059bf07495aacd9f5f0057823fc 2023-03-23 08:54:55,534 INFO [Listener at localhost/38371] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp/f1/7384f31bcba14a5f945cd7f188b71a62 2023-03-23 08:54:55,540 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp/f1/7384f31bcba14a5f945cd7f188b71a62 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62 2023-03-23 08:54:55,544 INFO [Listener at localhost/38371] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62, entries=1, sequenceid=5, filesize=4.9 K 2023-03-23 08:54:55,545 INFO [Listener at localhost/38371] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f36cd72431a8d4b98b336297a112e839 in 439ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:55,545 DEBUG [Listener at localhost/38371] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-23 08:54:55,546 DEBUG [Listener at localhost/38371] regionserver.HRegion(2446): Flush status journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:55,548 INFO [Listener at localhost/38371] regionserver.HRegion(2745): Flushing f36cd72431a8d4b98b336297a112e839 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-23 08:54:55,559 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303234b7228aabd7c4069951f64565cdf73d8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303234b7228aabd7c4069951f64565cdf73d8 2023-03-23 08:54:55,844 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-23 08:54:55,844 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-03-23 08:54:55,965 INFO [Listener at localhost/38371] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp/f1/5acf8686b94b4f65a1a539d8015ced03 2023-03-23 08:54:55,971 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp/f1/5acf8686b94b4f65a1a539d8015ced03 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03 2023-03-23 08:54:55,975 INFO [Listener at localhost/38371] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03, entries=1, sequenceid=9, filesize=4.9 K 2023-03-23 08:54:55,976 INFO [Listener at localhost/38371] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f36cd72431a8d4b98b336297a112e839 in 428ms, sequenceid=9, compaction requested=false 2023-03-23 08:54:55,977 DEBUG [Listener at localhost/38371] regionserver.HRegion(2446): Flush status journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:55,978 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-23 08:54:55,979 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-23 08:54:55,980 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-23 08:54:55,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:55,982 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562295982 2023-03-23 08:54:55,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:54:55,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-23 08:54:55,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-23 08:54:55,983 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x33e43777 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:54:55,993 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3219339e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:54:55,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230323 for table='testGetAfterCompaction' and column='f1' 2023-03-23 08:54:56,031 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303237368eff1136d4c15862062a631e4358e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303237368eff1136d4c15862062a631e4358e 2023-03-23 08:54:56,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testGetAfterCompaction' column='f1' 2023-03-23 08:54:56,032 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,032 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,050 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:54:56,051 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42016, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:54:56,058 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:54:56,059 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35828, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:54:56,073 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/f1/c314847d18464faf8a856ae62317592e first=Optional[r1] last=Optional[r2] 2023-03-23 08:54:56,075 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{f1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/f1/c314847d18464faf8a856ae62317592e}] 2023-03-23 08:54:56,079 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/f1/c314847d18464faf8a856ae62317592e for inclusion in f36cd72431a8d4b98b336297a112e839/f1 2023-03-23 08:54:56,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-23 08:54:56,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:54:56,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:56,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/f1/c314847d18464faf8a856ae62317592e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testGetAfterCompaction__r9ubkttf7isf19hcs941ii93fi9d48piikior9c5f3tatqhvq1fnq1pv3nfvk4tl/f1/c314847d18464faf8a856ae62317592e 2023-03-23 08:54:56,083 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,083 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testGetAfterCompaction__r9ubkttf7isf19hcs941ii93fi9d48piikior9c5f3tatqhvq1fnq1pv3nfvk4tl/f1/c314847d18464faf8a856ae62317592e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_ 2023-03-23 08:54:56,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testGetAfterCompaction__r9ubkttf7isf19hcs941ii93fi9d48piikior9c5f3tatqhvq1fnq1pv3nfvk4tl/f1/c314847d18464faf8a856ae62317592e into f36cd72431a8d4b98b336297a112e839/f1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_ - updating store file list. 2023-03-23 08:54:56,090 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_ into f36cd72431a8d4b98b336297a112e839/f1 2023-03-23 08:54:56,090 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testGetAfterCompaction__r9ubkttf7isf19hcs941ii93fi9d48piikior9c5f3tatqhvq1fnq1pv3nfvk4tl/f1/c314847d18464faf8a856ae62317592e into f36cd72431a8d4b98b336297a112e839/f1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_) 2023-03-23 08:54:56,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testGetAfterCompaction__r9ubkttf7isf19hcs941ii93fi9d48piikior9c5f3tatqhvq1fnq1pv3nfvk4tl/f1/c314847d18464faf8a856ae62317592e 2023-03-23 08:54:56,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(1912): f36cd72431a8d4b98b336297a112e839/f1 is initiating minor compaction 2023-03-23 08:54:56,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53e7be83; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:56,091 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f36cd72431a8d4b98b336297a112e839/f1 in testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:56,091 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp, totalSize=10.0 K 2023-03-23 08:54:56,091 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 5acf8686b94b4f65a1a539d8015ced03, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:54:56,091 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:54:56,093 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testGetAfterCompaction' column='f1' 2023-03-23 08:54:56,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:56,096 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303234b7228aabd7c4069951f64565cdf73d8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303234b7228aabd7c4069951f64565cdf73d8 2023-03-23 08:54:56,098 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230323e4e8d059bf07495aacd9f5f0057823fc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230323e4e8d059bf07495aacd9f5f0057823fc 2023-03-23 08:54:56,098 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-23 08:54:56,098 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230323' 2023-03-23 08:54:56,098 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230323 for table='testGetAfterCompaction' and column='f1' 2023-03-23 08:54:56,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-23 08:54:56,098 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:54:56,101 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): f36cd72431a8d4b98b336297a112e839#f1#compaction#77 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:54:56,104 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-23 08:54:56,106 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 125 msec 2023-03-23 08:54:56,134 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,134 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,134 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,141 DEBUG [Listener at localhost/38371] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-23 08:54:56,145 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-23 08:54:56,145 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=32923] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:56,146 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-23 08:54:56,146 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-23 08:54:56,146 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=32923] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-23 08:54:56,146 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=32923] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. because compaction request was cancelled 2023-03-23 08:54:56,197 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,198 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,198 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,249 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,249 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,249 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,300 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,300 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,300 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,351 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,351 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,351 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,402 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,402 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,402 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,453 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,453 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,453 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,494 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:54:56,504 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,504 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,504 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,555 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,555 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,555 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,606 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,606 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,606 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,657 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,657 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,657 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,708 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,708 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,708 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,759 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,759 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,759 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,810 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,810 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,810 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,861 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,861 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,861 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,912 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,912 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,912 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:56,951 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/.tmp/f1/43ccb4609f87482a862a7ba5a7a1fe93 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93 2023-03-23 08:54:56,955 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in f36cd72431a8d4b98b336297a112e839/f1 of f36cd72431a8d4b98b336297a112e839 into 43ccb4609f87482a862a7ba5a7a1fe93(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:54:56,955 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:56,956 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1679561696091; duration=0sec 2023-03-23 08:54:56,956 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:54:56,963 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679561696946; access_time=1679561696098; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,963 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,963 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:56,963 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:57,014 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679561696946; access_time=1679561696098; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:57,014 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695965; access_time=1679561695549; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:57,014 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:57,014 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679561696026; access_time=1679561696007; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-23 08:54:57,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_] to archive 2023-03-23 08:54:57,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:54:57,023 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/5acf8686b94b4f65a1a539d8015ced03 2023-03-23 08:54:57,024 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/a160f2c6512b401bbb8b8116ba03eabb_SeqId_12_ 2023-03-23 08:54:57,065 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679561696946; access_time=1679561696098; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:57,065 INFO [Listener at localhost/38371] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679561695534; access_time=1679561695109; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-23 08:54:57,184 INFO [Listener at localhost/38371] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-23 08:54:57,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-23 08:54:57,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,201 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561697201"}]},"ts":"1679561697201"} 2023-03-23 08:54:57,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-23 08:54:57,205 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-23 08:54:57,206 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-23 08:54:57,208 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, UNASSIGN}] 2023-03-23 08:54:57,209 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, UNASSIGN 2023-03-23 08:54:57,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f36cd72431a8d4b98b336297a112e839, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:57,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679561697210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561697210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561697210"}]},"ts":"1679561697210"} 2023-03-23 08:54:57,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure f36cd72431a8d4b98b336297a112e839, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:57,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-23 08:54:57,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:57,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f36cd72431a8d4b98b336297a112e839, disabling compactions & flushes 2023-03-23 08:54:57,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:57,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:57,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. after waiting 0 ms 2023-03-23 08:54:57,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:57,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-23 08:54:57,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-23 08:54:57,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839. 2023-03-23 08:54:57,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f36cd72431a8d4b98b336297a112e839: 2023-03-23 08:54:57,380 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:57,381 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f36cd72431a8d4b98b336297a112e839, regionState=CLOSED 2023-03-23 08:54:57,381 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679561697381"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561697381"}]},"ts":"1679561697381"} 2023-03-23 08:54:57,383 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-23 08:54:57,384 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure f36cd72431a8d4b98b336297a112e839, server=jenkins-hbase4.apache.org,32923,1679561616941 in 171 msec 2023-03-23 08:54:57,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-23 08:54:57,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f36cd72431a8d4b98b336297a112e839, UNASSIGN in 176 msec 2023-03-23 08:54:57,386 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561697386"}]},"ts":"1679561697386"} 2023-03-23 08:54:57,387 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-23 08:54:57,393 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-23 08:54:57,395 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 201 msec 2023-03-23 08:54:57,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-23 08:54:57,506 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-23 08:54:57,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-23 08:54:57,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,524 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,526 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-23 08:54:57,530 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:57,533 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1, FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/recovered.edits] 2023-03-23 08:54:57,537 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/43ccb4609f87482a862a7ba5a7a1fe93 2023-03-23 08:54:57,539 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/f1/7384f31bcba14a5f945cd7f188b71a62 2023-03-23 08:54:57,542 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/recovered.edits/15.seqid to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839/recovered.edits/15.seqid 2023-03-23 08:54:57,542 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testGetAfterCompaction/f36cd72431a8d4b98b336297a112e839 2023-03-23 08:54:57,542 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-23 08:54:57,543 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-23 08:54:57,544 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-23 08:54:57,548 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303237368eff1136d4c15862062a631e4358e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303237368eff1136d4c15862062a631e4358e 2023-03-23 08:54:57,549 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-23 08:54:57,552 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,584 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-23 08:54:57,592 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-23 08:54:57,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,594 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-23 08:54:57,594 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679561697594"}]},"ts":"9223372036854775807"} 2023-03-23 08:54:57,596 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-23 08:54:57,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f36cd72431a8d4b98b336297a112e839, NAME => 'testGetAfterCompaction,,1679561692984.f36cd72431a8d4b98b336297a112e839.', STARTKEY => '', ENDKEY => ''}] 2023-03-23 08:54:57,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-23 08:54:57,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1679561697596"}]},"ts":"9223372036854775807"} 2023-03-23 08:54:57,601 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-23 08:54:57,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-23 08:54:57,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 89 msec 2023-03-23 08:54:57,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-23 08:54:57,628 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-23 08:54:57,645 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=250 (was 246) - Thread LEAK? -, OpenFileDescriptor=633 (was 629) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=429 (was 413) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5746 (was 5764) 2023-03-23 08:54:57,659 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=250, OpenFileDescriptor=633, MaxFileDescriptor=60000, SystemLoadAverage=429, ProcessCount=169, AvailableMemoryMB=5745 2023-03-23 08:54:57,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-23 08:54:57,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-23 08:54:57,665 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:54:57,665 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-23 08:54:57,666 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:54:57,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-23 08:54:57,669 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:57,669 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:57,677 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:57,677 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 empty. 2023-03-23 08:54:57,678 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce empty. 2023-03-23 08:54:57,678 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb empty. 2023-03-23 08:54:57,678 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:57,678 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:57,678 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:57,678 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-23 08:54:57,695 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-23 08:54:57,697 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6469db6845c8da10a08493a71e41b196, NAME => 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:57,698 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => caf47a88030410666d2a8cb9e80797cb, NAME => 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:57,698 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 1da1b8621f257956854515270c9db5ce, NAME => 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:54:57,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-23 08:54:57,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing caf47a88030410666d2a8cb9e80797cb, disabling compactions & flushes 2023-03-23 08:54:58,124 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. after waiting 0 ms 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,124 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,124 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 6469db6845c8da10a08493a71e41b196, disabling compactions & flushes 2023-03-23 08:54:58,124 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. after waiting 0 ms 2023-03-23 08:54:58,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,125 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,125 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:54:58,138 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 1da1b8621f257956854515270c9db5ce, disabling compactions & flushes 2023-03-23 08:54:58,139 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. after waiting 0 ms 2023-03-23 08:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,139 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,139 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:54:58,143 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:54:58,144 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561698144"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561698144"}]},"ts":"1679561698144"} 2023-03-23 08:54:58,144 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698144"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561698144"}]},"ts":"1679561698144"} 2023-03-23 08:54:58,144 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698144"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561698144"}]},"ts":"1679561698144"} 2023-03-23 08:54:58,147 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-23 08:54:58,148 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:54:58,148 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561698148"}]},"ts":"1679561698148"} 2023-03-23 08:54:58,149 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-23 08:54:58,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN}] 2023-03-23 08:54:58,156 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN 2023-03-23 08:54:58,156 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN 2023-03-23 08:54:58,156 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN 2023-03-23 08:54:58,156 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:58,156 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:58,156 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:54:58,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-23 08:54:58,309 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,309 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,309 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698309"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561698309"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561698309"}]},"ts":"1679561698309"} 2023-03-23 08:54:58,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698309"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561698309"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561698309"}]},"ts":"1679561698309"} 2023-03-23 08:54:58,309 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561698309"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561698309"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561698309"}]},"ts":"1679561698309"} 2023-03-23 08:54:58,311 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:58,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:58,314 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:54:58,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1da1b8621f257956854515270c9db5ce, NAME => 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.', STARTKEY => '2', ENDKEY => ''} 2023-03-23 08:54:58,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,469 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,472 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:54:58,472 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:54:58,473 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1da1b8621f257956854515270c9db5ce columnFamilyName family1 2023-03-23 08:54:58,476 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(310): Store=1da1b8621f257956854515270c9db5ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,477 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,478 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2 2023-03-23 08:54:58,478 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2 2023-03-23 08:54:58,479 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1da1b8621f257956854515270c9db5ce columnFamilyName family2 2023-03-23 08:54:58,480 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(310): Store=1da1b8621f257956854515270c9db5ce/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:58,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:54:58,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:58,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1da1b8621f257956854515270c9db5ce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66425841, jitterRate=-0.010177835822105408}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:58,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:54:58,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., pid=84, masterSystemTime=1679561698462 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => caf47a88030410666d2a8cb9e80797cb, NAME => 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-23 08:54:58,491 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,491 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698491"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561698491"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561698491"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561698491"}]},"ts":"1679561698491"} 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,493 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,494 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:54:58,494 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:54:58,495 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region caf47a88030410666d2a8cb9e80797cb columnFamilyName family1 2023-03-23 08:54:58,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-03-23 08:54:58,495 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941 in 182 msec 2023-03-23 08:54:58,496 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN in 341 msec 2023-03-23 08:54:58,496 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(310): Store=caf47a88030410666d2a8cb9e80797cb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,496 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,498 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2 2023-03-23 08:54:58,498 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2 2023-03-23 08:54:58,498 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region caf47a88030410666d2a8cb9e80797cb columnFamilyName family2 2023-03-23 08:54:58,499 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(310): Store=caf47a88030410666d2a8cb9e80797cb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:58,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:54:58,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:58,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened caf47a88030410666d2a8cb9e80797cb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72770982, jitterRate=0.0843721330165863}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:58,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:54:58,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., pid=85, masterSystemTime=1679561698462 2023-03-23 08:54:58,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6469db6845c8da10a08493a71e41b196, NAME => 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.', STARTKEY => '', ENDKEY => '1'} 2023-03-23 08:54:58,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,512 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:54:58,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,513 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561698512"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561698512"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561698512"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561698512"}]},"ts":"1679561698512"} 2023-03-23 08:54:58,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,514 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,516 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:54:58,516 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:54:58,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-03-23 08:54:58,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941 in 201 msec 2023-03-23 08:54:58,516 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6469db6845c8da10a08493a71e41b196 columnFamilyName family1 2023-03-23 08:54:58,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN in 364 msec 2023-03-23 08:54:58,517 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(310): Store=6469db6845c8da10a08493a71e41b196/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,518 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,519 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2 2023-03-23 08:54:58,519 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2 2023-03-23 08:54:58,519 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6469db6845c8da10a08493a71e41b196 columnFamilyName family2 2023-03-23 08:54:58,520 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(310): Store=6469db6845c8da10a08493a71e41b196/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:54:58,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:54:58,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:54:58,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:54:58,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6469db6845c8da10a08493a71e41b196; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62553690, jitterRate=-0.0678773820400238}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:54:58,526 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:54:58,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., pid=83, masterSystemTime=1679561698462 2023-03-23 08:54:58,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,528 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,528 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561698528"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561698528"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561698528"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561698528"}]},"ts":"1679561698528"} 2023-03-23 08:54:58,530 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-03-23 08:54:58,530 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941 in 218 msec 2023-03-23 08:54:58,531 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-23 08:54:58,532 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN in 378 msec 2023-03-23 08:54:58,532 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:54:58,532 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561698532"}]},"ts":"1679561698532"} 2023-03-23 08:54:58,533 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-23 08:54:58,536 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:54:58,537 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 873 msec 2023-03-23 08:54:58,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-23 08:54:58,771 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-23 08:54:58,780 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:54:58,782 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:54:58,782 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:54:58,787 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:54:58,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-23 08:54:58,791 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-23 08:54:58,791 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:58,791 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-23 08:54:58,791 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-23 08:54:58,792 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:54:58,792 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:54:58,793 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:54:58,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:54:58,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:54:58,793 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:54:58,794 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:54:58,794 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:54:58,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:54:58,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:54:58,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:54:58,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-23 08:54:58,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-23 08:54:58,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-23 08:54:58,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:54:58,795 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-23 08:54:58,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:58,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:54:58,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:58,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:54:58,796 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:54:58,797 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6469db6845c8da10a08493a71e41b196 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:54:58,797 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:54:58,797 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:54:58,797 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1da1b8621f257956854515270c9db5ce 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:54:58,797 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:54:58,799 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing caf47a88030410666d2a8cb9e80797cb 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:54:58,869 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a 2023-03-23 08:54:59,247 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 2023-03-23 08:54:59,271 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e 2023-03-23 08:54:59,284 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/1310ee368980452198ba3bbb20ce6aa3 2023-03-23 08:54:59,287 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/c7580ba1bc7d4778a981d28b8c652243 2023-03-23 08:54:59,304 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17 2023-03-23 08:54:59,318 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/6eac470c41cd48cd836ec110e27b664f 2023-03-23 08:54:59,323 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/1310ee368980452198ba3bbb20ce6aa3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3 2023-03-23 08:54:59,329 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:54:59,330 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/6eac470c41cd48cd836ec110e27b664f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f 2023-03-23 08:54:59,333 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:54:59,334 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6469db6845c8da10a08493a71e41b196 in 537ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:59,334 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-23 08:54:59,335 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:54:59,335 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:54:59,335 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:54:59,653 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/4466553c64d749998af888c77372563b 2023-03-23 08:54:59,705 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0 2023-03-23 08:54:59,711 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 2023-03-23 08:54:59,716 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/c7580ba1bc7d4778a981d28b8c652243 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243 2023-03-23 08:54:59,721 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:54:59,722 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 2023-03-23 08:54:59,728 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:54:59,729 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1da1b8621f257956854515270c9db5ce in 932ms, sequenceid=5, compaction requested=false 2023-03-23 08:54:59,732 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:54:59,732 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:54:59,732 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:00,071 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72 2023-03-23 08:55:00,081 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/a406ffd2aeee4b648dff9d910f16beea 2023-03-23 08:55:00,090 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/4466553c64d749998af888c77372563b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b 2023-03-23 08:55:00,097 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:55:00,097 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/a406ffd2aeee4b648dff9d910f16beea as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea 2023-03-23 08:55:00,103 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:00,105 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for caf47a88030410666d2a8cb9e80797cb in 1305ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:00,105 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:00,105 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:00,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:00,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:00,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-23 08:55:00,105 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-23 08:55:00,107 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,107 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:00,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:00,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,109 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:00,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-23 08:55:00,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35376f1f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:00,110 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-23 08:55:00,110 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,112 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,112 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-23 08:55:00,112 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-23 08:55:00,112 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:00,112 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,112 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:00,114 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:00,115 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-23 08:55:00,115 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:00,115 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-23 08:55:00,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:00,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:00,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-23 08:55:00,119 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-23 08:55:00,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:00,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:00,119 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-23 08:55:00,119 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:00,122 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,122 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:00,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:00,122 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:00,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:00,123 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:00,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,124 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:00,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:00,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,127 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:00,127 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:00,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:00,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,130 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:00,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-23 08:55:00,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:00,130 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:00,231 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:00,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:00,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:00,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:00,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:00,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:00,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:00,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-23 08:55:00,238 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-23 08:55:00,238 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:00,239 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-23 08:55:00,239 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-23 08:55:00,239 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,239 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:00,241 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:00,241 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:00,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:00,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:00,241 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:00,241 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:00,242 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:00,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:00,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:00,242 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-23 08:55:00,242 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-23 08:55:00,242 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-23 08:55:00,242 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-23 08:55:00,243 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:00,243 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-23 08:55:00,244 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:00,244 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:55:00,244 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6469db6845c8da10a08493a71e41b196 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:00,254 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:00,254 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:00,254 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:00,254 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:55:00,255 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing caf47a88030410666d2a8cb9e80797cb 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:00,254 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:00,254 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:55:00,255 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1da1b8621f257956854515270c9db5ce 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:00,703 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a 2023-03-23 08:55:00,709 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/2caf681daca6482a8a91e6719e7f2d54 2023-03-23 08:55:00,713 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 2023-03-23 08:55:00,716 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f 2023-03-23 08:55:00,721 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/f3c0d0d20baa4ea7bab197f5a521cb04 2023-03-23 08:55:00,729 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/a0e51d5671ae42de98d6b5c0a0580d37 2023-03-23 08:55:01,151 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696 2023-03-23 08:55:01,155 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d 2023-03-23 08:55:01,181 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/eb4ceef45c04496c925124fb07d5e86d 2023-03-23 08:55:01,182 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd 2023-03-23 08:55:01,188 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/f3c0d0d20baa4ea7bab197f5a521cb04 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04 2023-03-23 08:55:01,192 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:01,193 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/eb4ceef45c04496c925124fb07d5e86d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d 2023-03-23 08:55:01,198 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:01,199 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1da1b8621f257956854515270c9db5ce in 944ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:01,199 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:01,199 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:01,199 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:01,590 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/7f22b77db3614734af37d10795d0f832 2023-03-23 08:55:01,595 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/2caf681daca6482a8a91e6719e7f2d54 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54 2023-03-23 08:55:01,595 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/287701c665e84e268ec9aa86a1fc086b 2023-03-23 08:55:01,600 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:01,601 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/7f22b77db3614734af37d10795d0f832 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832 2023-03-23 08:55:01,602 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/a0e51d5671ae42de98d6b5c0a0580d37 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37 2023-03-23 08:55:01,609 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:01,609 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:01,610 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/287701c665e84e268ec9aa86a1fc086b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b 2023-03-23 08:55:01,610 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6469db6845c8da10a08493a71e41b196 in 1366ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:01,610 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:01,610 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:01,610 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:01,614 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:01,614 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for caf47a88030410666d2a8cb9e80797cb in 1359ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:01,615 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:01,615 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-23 08:55:01,615 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-23 08:55:01,616 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:01,617 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,617 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:01,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-23 08:55:01,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c111654[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:01,618 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-23 08:55:01,618 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,619 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,620 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-23 08:55:01,620 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-23 08:55:01,620 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:01,620 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,620 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:01,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-23 08:55:01,622 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:01,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:01,623 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-23 08:55:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:01,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:01,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-23 08:55:01,625 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-23 08:55:01,625 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:01,625 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:01,625 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-23 08:55:01,625 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:01,627 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,627 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:01,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:01,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,627 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:01,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:01,627 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:01,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:01,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:01,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:01,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,630 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:01,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:01,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-23 08:55:01,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:01,632 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:01,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,633 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:01,633 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:01,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:01,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:01,733 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:01,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:01,736 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:01,736 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:01,736 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:01,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:01,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:01,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-23 08:55:01,742 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-23 08:55:01,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:01,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-23 08:55:01,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-23 08:55:01,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,743 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:01,744 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,744 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:01,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:01,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:01,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:01,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:01,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:01,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-23 08:55:01,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-23 08:55:01,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-23 08:55:01,746 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-23 08:55:01,746 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:01,746 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-23 08:55:01,746 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:01,746 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:01,746 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:01,746 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:55:01,746 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:01,746 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:01,747 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:55:01,746 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:55:01,747 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6469db6845c8da10a08493a71e41b196 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:01,747 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing caf47a88030410666d2a8cb9e80797cb 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:01,747 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1da1b8621f257956854515270c9db5ce 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:01,778 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 2023-03-23 08:55:01,781 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 2023-03-23 08:55:01,788 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/10d486d59b3d4ab981afd70ee16f96c2 2023-03-23 08:55:01,797 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/1cda00ec17d5478c843ba62cb80f15bb 2023-03-23 08:55:01,809 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc 2023-03-23 08:55:02,175 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 2023-03-23 08:55:02,218 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/f4296c5012e64ac8b5e771cad757469f 2023-03-23 08:55:02,219 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d 2023-03-23 08:55:02,222 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/10d486d59b3d4ab981afd70ee16f96c2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2 2023-03-23 08:55:02,225 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 2023-03-23 08:55:02,226 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:02,227 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/f4296c5012e64ac8b5e771cad757469f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f 2023-03-23 08:55:02,230 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/1cda00ec17d5478c843ba62cb80f15bb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb 2023-03-23 08:55:02,232 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:02,233 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1da1b8621f257956854515270c9db5ce in 486ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:02,233 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:02,233 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:02,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:02,235 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:02,235 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 2023-03-23 08:55:02,239 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:02,240 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6469db6845c8da10a08493a71e41b196 in 494ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:02,240 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:02,240 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:02,240 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:02,581 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/033a0585ec65432280f909c048b2a09c 2023-03-23 08:55:02,596 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d 2023-03-23 08:55:02,600 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/708212eed5214c2382a1633ffb2a1f42 2023-03-23 08:55:02,605 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/033a0585ec65432280f909c048b2a09c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c 2023-03-23 08:55:02,609 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:02,610 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/708212eed5214c2382a1633ffb2a1f42 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42 2023-03-23 08:55:02,615 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:02,616 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for caf47a88030410666d2a8cb9e80797cb in 869ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:02,616 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:02,616 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-23 08:55:02,616 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-23 08:55:02,619 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,619 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:02,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:02,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:02,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:02,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-23 08:55:02,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d00b590[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:02,621 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-23 08:55:02,621 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,622 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-23 08:55:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-23 08:55:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:02,623 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,623 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-23 08:55:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:02,624 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-23 08:55:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:02,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:02,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-23 08:55:02,627 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-23 08:55:02,627 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:02,627 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:02,627 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-23 08:55:02,627 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:02,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:02,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,632 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:02,632 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,634 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:02,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,636 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:02,636 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:02,637 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-23 08:55:02,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:02,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:02,637 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:02,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-23 08:55:02,738 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:02,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:02,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:02,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:02,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:02,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:02,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:02,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-23 08:55:02,744 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-23 08:55:02,744 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:02,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-23 08:55:02,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-23 08:55:02,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,745 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:02,746 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:02,746 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:02,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:02,747 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:02,747 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:02,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:02,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:02,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:02,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:02,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-23 08:55:02,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-23 08:55:02,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-23 08:55:02,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-23 08:55:02,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:02,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:02,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:02,748 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:02,748 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6469db6845c8da10a08493a71e41b196 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:55:02,749 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing caf47a88030410666d2a8cb9e80797cb 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:02,748 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:55:02,749 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1da1b8621f257956854515270c9db5ce 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:02,798 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c 2023-03-23 08:55:02,800 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 2023-03-23 08:55:02,827 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/14a964415ddb487aa266f359e7a82522 2023-03-23 08:55:02,880 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:03,195 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 2023-03-23 08:55:03,229 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 2023-03-23 08:55:03,244 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088 2023-03-23 08:55:03,600 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/317ffa2f61ee4360963349cc7a6188d9 2023-03-23 08:55:03,641 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d 2023-03-23 08:55:03,648 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/fcc0807b1af94e7aaefe81a62253caeb 2023-03-23 08:55:03,652 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/14a964415ddb487aa266f359e7a82522 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522 2023-03-23 08:55:03,655 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:03,655 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/fcc0807b1af94e7aaefe81a62253caeb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb 2023-03-23 08:55:03,658 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:03,659 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6469db6845c8da10a08493a71e41b196 in 911ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:03,659 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:03,659 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:03,659 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:04,013 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42 2023-03-23 08:55:04,045 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 2023-03-23 08:55:04,049 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 2023-03-23 08:55:04,052 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:04,053 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 2023-03-23 08:55:04,056 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:04,057 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1da1b8621f257956854515270c9db5ce in 1308ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:04,057 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:04,057 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:04,057 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:04,418 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/f96299eaba8b4f74b1bcd32b38950406 2023-03-23 08:55:04,422 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/317ffa2f61ee4360963349cc7a6188d9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9 2023-03-23 08:55:04,425 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:04,425 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/f96299eaba8b4f74b1bcd32b38950406 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406 2023-03-23 08:55:04,429 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:04,429 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for caf47a88030410666d2a8cb9e80797cb in 1680ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:04,429 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:04,429 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-23 08:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-23 08:55:04,431 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,431 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:04,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:04,431 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,431 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:04,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:04,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:04,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:04,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-23 08:55:04,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ea0ef88[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:04,433 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-23 08:55:04,433 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,434 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,434 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-23 08:55:04,434 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-23 08:55:04,434 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:04,434 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,435 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:04,436 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,436 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-23 08:55:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,436 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:04,436 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-23 08:55:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:04,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:04,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:04,438 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-23 08:55:04,438 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:04,438 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:04,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-23 08:55:04,438 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-23 08:55:04,438 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:04,439 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,439 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:04,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,440 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:04,440 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,441 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:04,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:04,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:04,443 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:04,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-23 08:55:04,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-23 08:55:04,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:04,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:04,443 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:04,443 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:04,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:04,544 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:04,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:04,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:04,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679561704554 (current time:1679561704554). 2023-03-23 08:55:04,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-23 08:55:04,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] zookeeper.ReadOnlyZKClient(139): Connect 0x12f72c10 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:04,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@e7f4bec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:04,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:04,569 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52098, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:04,569 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x12f72c10 to 127.0.0.1:50031 2023-03-23 08:55:04,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:04,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-23 08:55:04,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:04,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:04,579 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:04,580 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679562304580 2023-03-23 08:55:04,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:04,580 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1679561704545 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-23 08:55:04,581 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:04,581 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-23 08:55:04,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-03-23 08:55:04,582 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-23 08:55:04,583 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1679562304583 2023-03-23 08:55:04,585 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:04,585 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-23 08:55:04,685 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:04,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:04,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:04,689 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-23 08:55:04,889 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:04,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:04,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:04,890 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-23 08:55:05,000 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1679561704545 2023-03-23 08:55:05,001 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1679561704545' 2023-03-23 08:55:05,001 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-23 08:55:05,001 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1679561704545' starting 'acquire' 2023-03-23 08:55:05,001 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1679561704545', kicking off acquire phase on members. 2023-03-23 08:55:05,001 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,001 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1679561704545 2023-03-23 08:55:05,002 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-23 08:55:05,002 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-23 08:55:05,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-23 08:55:05,003 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,003 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1679561704545 2023-03-23 08:55:05,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-23 08:55:05,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1679561704545 2023-03-23 08:55:05,003 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1679561704545 from table testCompactionWithHFileLink type FLUSH 2023-03-23 08:55:05,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1679561704545 2023-03-23 08:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1679561704545' with timeout 300000ms 2023-03-23 08:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-23 08:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1679561704545' starting 'acquire' stage 2023-03-23 08:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1679561704545' locally acquired 2023-03-23 08:55:05,006 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (snaptb-1679561704545) in zk 2023-03-23 08:55:05,007 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-23 08:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:05,007 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1679561704545' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:05,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'snaptb-1679561704545' on coordinator 2023-03-23 08:55:05,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70e138d4[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:05,009 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1679561704545' starting 'in-barrier' execution. 2023-03-23 08:55:05,009 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,010 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,011 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1679561704545' received 'reached' from coordinator. 2023-03-23 08:55:05,011 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,011 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:05,011 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:05,011 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:05,012 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:05,012 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:05,012 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:05,013 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.' region-info for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,013 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.' region-info for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,013 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.' region-info for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,018 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-23 08:55:05,018 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-23 08:55:05,018 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9] hfiles 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522] hfiles 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2] hfiles 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406] hfiles 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb] hfiles 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4] hfiles 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,025 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,026 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406 for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,041 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. completed. 2023-03-23 08:55:05,041 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:05,042 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-23 08:55:05,190 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:05,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:05,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:05,191 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-23 08:55:05,433 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. completed. 2023-03-23 08:55:05,433 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:05,433 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-23 08:55:05,438 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. completed. 2023-03-23 08:55:05,438 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-23 08:55:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-23 08:55:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1679561704545' locally completed 2023-03-23 08:55:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1679561704545' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:05,440 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1679561704545' has notified controller of completion 2023-03-23 08:55:05,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:05,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-23 08:55:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1679561704545' completed. 2023-03-23 08:55:05,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:05,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1679561704545' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:05,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'snaptb-1679561704545', counting down latch. Waiting for 0 more 2023-03-23 08:55:05,442 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1679561704545' execution completed 2023-03-23 08:55:05,442 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:05,442 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:05,442 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1679561704545 2023-03-23 08:55:05,442 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1679561704545including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-23 08:55:05,444 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,444 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,444 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-23 08:55:05,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-23 08:55:05,444 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,445 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679561704545 2023-03-23 08:55:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679561704545 2023-03-23 08:55:05,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-23 08:55:05,447 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679561704545 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679561704545/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,447 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1679561704545 2023-03-23 08:55:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-23 08:55:05,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1679561704545 because node does not exist (not an error) 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679561704545 2023-03-23 08:55:05,447 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-23 08:55:05,447 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679561704545 2023-03-23 08:55:05,447 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1679561704545 2023-03-23 08:55:05,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-23 08:55:05,447 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-23 08:55:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a] mob files 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c 2023-03-23 08:55:05,449 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696] mob files 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d 2023-03-23 08:55:05,450 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696 2023-03-23 08:55:05,691 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:05,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:05,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:05,692 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-23 08:55:05,844 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-23 08:55:05,855 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1679561704545 2023-03-23 08:55:05,857 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/snaptb-1679561704545 2023-03-23 08:55:06,295 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/snaptb-1679561704545 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/snaptb-1679561704545 2023-03-23 08:55:06,692 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:06,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:06,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:06,693 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-03-23 08:55:07,133 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1679561704545 of table testCompactionWithHFileLink completed 2023-03-23 08:55:07,133 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/snaptb-1679561704545 2023-03-23 08:55:07,134 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/snaptb-1679561704545 2023-03-23 08:55:07,134 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1679561704545 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679561704580Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1679561705855 (+1275 ms)Consolidate snapshot: snaptb-1679561704545 at 1679561705855Loading Region manifests for snaptb-1679561704545 at 1679561705855Writing data manifest for snaptb-1679561704545 at 1679561705864 (+9 ms)Verifying snapshot: snaptb-1679561704545 at 1679561706270 (+406 ms)Snapshot snaptb-1679561704545 of table testCompactionWithHFileLink completed at 1679561707133 (+863 ms) 2023-03-23 08:55:07,135 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-23 08:55:07,136 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.5540 sec 2023-03-23 08:55:08,694 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:08,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:08,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-23 08:55:08,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:08,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:08,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-23 08:55:08,725 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-23 08:55:08,725 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:08,725 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-23 08:55:08,725 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-23 08:55:08,725 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:08,725 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:08,726 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:08,726 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:08,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:08,726 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:08,727 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-23 08:55:08,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-23 08:55:08,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-23 08:55:08,727 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-23 08:55:08,727 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:08,727 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:08,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. started... 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. started... 2023-03-23 08:55:08,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. started... 2023-03-23 08:55:08,728 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 1da1b8621f257956854515270c9db5ce 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:08,728 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6469db6845c8da10a08493a71e41b196 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:08,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:08,728 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing caf47a88030410666d2a8cb9e80797cb 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:09,548 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/bddf2546bdad423b99046e33e431e411 2023-03-23 08:55:09,550 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/188ed28964904f11ba064bf7f1906ff6 2023-03-23 08:55:09,551 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/ad8dd7719c42421491817d351f3d2a50 2023-03-23 08:55:09,554 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bddf2546bdad423b99046e33e431e411 2023-03-23 08:55:09,555 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 188ed28964904f11ba064bf7f1906ff6 2023-03-23 08:55:09,555 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ad8dd7719c42421491817d351f3d2a50 2023-03-23 08:55:09,600 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/1171c53aaa4244c193ce9bd582f25368 2023-03-23 08:55:09,604 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1171c53aaa4244c193ce9bd582f25368 2023-03-23 08:55:09,605 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/188ed28964904f11ba064bf7f1906ff6 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6 2023-03-23 08:55:09,618 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 188ed28964904f11ba064bf7f1906ff6 2023-03-23 08:55:09,618 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6, entries=3, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:09,619 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/1171c53aaa4244c193ce9bd582f25368 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368 2023-03-23 08:55:09,622 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/31b9267f3aa9465a9c8e6098bd74e7d1 2023-03-23 08:55:09,624 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1171c53aaa4244c193ce9bd582f25368 2023-03-23 08:55:09,625 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368, entries=1, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:09,625 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6469db6845c8da10a08493a71e41b196 in 897ms, sequenceid=24, compaction requested=true 2023-03-23 08:55:09,626 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:09,626 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:09,626 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:09,626 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31b9267f3aa9465a9c8e6098bd74e7d1 2023-03-23 08:55:09,626 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/ad8dd7719c42421491817d351f3d2a50 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50 2023-03-23 08:55:09,637 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ad8dd7719c42421491817d351f3d2a50 2023-03-23 08:55:09,637 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50, entries=3, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:09,641 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/31b9267f3aa9465a9c8e6098bd74e7d1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1 2023-03-23 08:55:09,650 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31b9267f3aa9465a9c8e6098bd74e7d1 2023-03-23 08:55:09,650 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1, entries=1, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:09,651 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for caf47a88030410666d2a8cb9e80797cb in 923ms, sequenceid=24, compaction requested=true 2023-03-23 08:55:09,651 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:09,651 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:09,651 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:10,002 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/ab3fb5fde458474d91b7b4f26d690354 2023-03-23 08:55:10,007 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3fb5fde458474d91b7b4f26d690354 2023-03-23 08:55:10,007 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/bddf2546bdad423b99046e33e431e411 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411 2023-03-23 08:55:10,013 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bddf2546bdad423b99046e33e431e411 2023-03-23 08:55:10,013 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411, entries=3, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:10,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/ab3fb5fde458474d91b7b4f26d690354 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354 2023-03-23 08:55:10,021 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab3fb5fde458474d91b7b4f26d690354 2023-03-23 08:55:10,021 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354, entries=1, sequenceid=24, filesize=5.0 K 2023-03-23 08:55:10,022 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1da1b8621f257956854515270c9db5ce in 1294ms, sequenceid=24, compaction requested=true 2023-03-23 08:55:10,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:10,022 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:10,022 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:10,022 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:10,022 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,022 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-23 08:55:10,022 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-23 08:55:10,024 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,024 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:10,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-23 08:55:10,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6456232a[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:10,027 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-23 08:55:10,027 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,028 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-23 08:55:10,028 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,028 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-23 08:55:10,028 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-23 08:55:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:10,033 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:10,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-23 08:55:10,036 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-23 08:55:10,036 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:10,036 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:10,036 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-23 08:55:10,036 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:10,038 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,038 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:10,038 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:10,038 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,040 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-23 08:55:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-23 08:55:10,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:10,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-23 08:55:10,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:10,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:10,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:10,042 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:10,143 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:10,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:10,144 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:10,144 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 6469db6845c8da10a08493a71e41b196/family1 is initiating major compaction (all files) 2023-03-23 08:55:10,144 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:10,145 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:10,145 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 6469db6845c8da10a08493a71e41b196/family1 in testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:10,146 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp, totalSize=26.7 K 2023-03-23 08:55:10,146 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1310ee368980452198ba3bbb20ce6aa3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698780 2023-03-23 08:55:10,147 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 2caf681daca6482a8a91e6719e7f2d54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:10,147 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1cda00ec17d5478c843ba62cb80f15bb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:10,147 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 14a964415ddb487aa266f359e7a82522, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:10,148 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 188ed28964904f11ba064bf7f1906ff6, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:10,161 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 6469db6845c8da10a08493a71e41b196#family1#compaction#108 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:10,176 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del 2023-03-23 08:55:10,176 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del 2023-03-23 08:55:10,195 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98d295c566a44f67a1326ceed6ce82b7 2023-03-23 08:55:10,195 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/98d295c566a44f67a1326ceed6ce82b7 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/98d295c566a44f67a1326ceed6ce82b7 2023-03-23 08:55:10,201 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 98d295c566a44f67a1326ceed6ce82b7 2023-03-23 08:55:10,202 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6469db6845c8da10a08493a71e41b196/family1 of 6469db6845c8da10a08493a71e41b196 into 98d295c566a44f67a1326ceed6ce82b7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:10,202 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:10,202 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:10,202 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 6469db6845c8da10a08493a71e41b196/family2 is initiating major compaction (all files) 2023-03-23 08:55:10,202 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:10,202 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:10,203 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 6469db6845c8da10a08493a71e41b196/family2 in testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:10,203 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp, totalSize=25.4 K 2023-03-23 08:55:10,204 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 6eac470c41cd48cd836ec110e27b664f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698780 2023-03-23 08:55:10,204 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 7f22b77db3614734af37d10795d0f832, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:10,204 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 2182fa1a0d724cdcbe69be3b24d5aaa9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:10,205 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting fcc0807b1af94e7aaefe81a62253caeb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:10,206 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1171c53aaa4244c193ce9bd582f25368, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:10,220 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 6469db6845c8da10a08493a71e41b196#family2#compaction#109 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:10,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6] to archive 2023-03-23 08:55:10,225 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:10,229 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1310ee368980452198ba3bbb20ce6aa3 2023-03-23 08:55:10,230 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/2caf681daca6482a8a91e6719e7f2d54 2023-03-23 08:55:10,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/1cda00ec17d5478c843ba62cb80f15bb 2023-03-23 08:55:10,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/14a964415ddb487aa266f359e7a82522 2023-03-23 08:55:10,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/188ed28964904f11ba064bf7f1906ff6 2023-03-23 08:55:10,643 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del 2023-03-23 08:55:10,644 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del 2023-03-23 08:55:11,347 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:11,459 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bca642edc6d74b5db4c91ffa16f1769c 2023-03-23 08:55:11,459 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/bca642edc6d74b5db4c91ffa16f1769c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/bca642edc6d74b5db4c91ffa16f1769c 2023-03-23 08:55:11,463 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bca642edc6d74b5db4c91ffa16f1769c 2023-03-23 08:55:11,464 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6469db6845c8da10a08493a71e41b196/family2 of 6469db6845c8da10a08493a71e41b196 into bca642edc6d74b5db4c91ffa16f1769c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:11,464 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:11,465 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:11,465 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 1da1b8621f257956854515270c9db5ce/family1 is initiating major compaction (all files) 2023-03-23 08:55:11,465 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:11,465 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:11,466 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 1da1b8621f257956854515270c9db5ce/family1 in testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:11,466 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp, totalSize=26.7 K 2023-03-23 08:55:11,466 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting c7580ba1bc7d4778a981d28b8c652243, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:11,466 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting f3c0d0d20baa4ea7bab197f5a521cb04, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:11,467 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 10d486d59b3d4ab981afd70ee16f96c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:11,467 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 5d7c3fa6815248dfbc446d05d5dfb8f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:11,467 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting bddf2546bdad423b99046e33e431e411, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:11,479 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 1da1b8621f257956854515270c9db5ce#family1#compaction#110 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:11,503 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del 2023-03-23 08:55:11,503 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del 2023-03-23 08:55:11,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368] to archive 2023-03-23 08:55:11,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:11,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/6eac470c41cd48cd836ec110e27b664f 2023-03-23 08:55:11,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/7f22b77db3614734af37d10795d0f832 2023-03-23 08:55:11,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/2182fa1a0d724cdcbe69be3b24d5aaa9 2023-03-23 08:55:11,528 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/fcc0807b1af94e7aaefe81a62253caeb 2023-03-23 08:55:11,531 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/1171c53aaa4244c193ce9bd582f25368 2023-03-23 08:55:12,329 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5bb573a08514599a6decb4b7a18ae9e 2023-03-23 08:55:12,330 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/a5bb573a08514599a6decb4b7a18ae9e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/a5bb573a08514599a6decb4b7a18ae9e 2023-03-23 08:55:12,338 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5bb573a08514599a6decb4b7a18ae9e 2023-03-23 08:55:12,340 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1da1b8621f257956854515270c9db5ce/family1 of 1da1b8621f257956854515270c9db5ce into a5bb573a08514599a6decb4b7a18ae9e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:12,340 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:12,340 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:12,340 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 1da1b8621f257956854515270c9db5ce/family2 is initiating major compaction (all files) 2023-03-23 08:55:12,340 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:12,340 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:12,340 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 1da1b8621f257956854515270c9db5ce/family2 in testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:12,340 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp, totalSize=25.4 K 2023-03-23 08:55:12,341 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0d2ffc4d5cfa4e538a8b3f96c2647908, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:12,341 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting eb4ceef45c04496c925124fb07d5e86d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:12,341 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting f4296c5012e64ac8b5e771cad757469f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:12,342 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting e2cb4c734e224bce9e4c8cc16e3e31c4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:12,342 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting ab3fb5fde458474d91b7b4f26d690354, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:12,355 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 1da1b8621f257956854515270c9db5ce#family2#compaction#111 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:12,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411] to archive 2023-03-23 08:55:12,428 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:12,430 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/c7580ba1bc7d4778a981d28b8c652243 2023-03-23 08:55:12,431 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/f3c0d0d20baa4ea7bab197f5a521cb04 2023-03-23 08:55:12,432 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/10d486d59b3d4ab981afd70ee16f96c2 2023-03-23 08:55:12,437 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/5d7c3fa6815248dfbc446d05d5dfb8f2 2023-03-23 08:55:12,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/bddf2546bdad423b99046e33e431e411 2023-03-23 08:55:12,763 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del 2023-03-23 08:55:12,764 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del 2023-03-23 08:55:13,580 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80f9a9af932f4461b3e5ea16a1098cb3 2023-03-23 08:55:13,581 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/80f9a9af932f4461b3e5ea16a1098cb3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/80f9a9af932f4461b3e5ea16a1098cb3 2023-03-23 08:55:13,585 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80f9a9af932f4461b3e5ea16a1098cb3 2023-03-23 08:55:13,586 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1da1b8621f257956854515270c9db5ce/family2 of 1da1b8621f257956854515270c9db5ce into 80f9a9af932f4461b3e5ea16a1098cb3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:13,586 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:13,586 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:13,586 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): caf47a88030410666d2a8cb9e80797cb/family1 is initiating major compaction (all files) 2023-03-23 08:55:13,587 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:13,587 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:13,587 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of caf47a88030410666d2a8cb9e80797cb/family1 in testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:13,587 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp, totalSize=26.7 K 2023-03-23 08:55:13,587 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 4466553c64d749998af888c77372563b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:13,588 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting a0e51d5671ae42de98d6b5c0a0580d37, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:13,589 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 033a0585ec65432280f909c048b2a09c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:13,589 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 317ffa2f61ee4360963349cc7a6188d9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:13,590 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting ad8dd7719c42421491817d351f3d2a50, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:13,620 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): caf47a88030410666d2a8cb9e80797cb#family1#compaction#112 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:13,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354] to archive 2023-03-23 08:55:13,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:13,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/0d2ffc4d5cfa4e538a8b3f96c2647908 2023-03-23 08:55:13,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/eb4ceef45c04496c925124fb07d5e86d 2023-03-23 08:55:13,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/f4296c5012e64ac8b5e771cad757469f 2023-03-23 08:55:13,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/e2cb4c734e224bce9e4c8cc16e3e31c4 2023-03-23 08:55:13,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ab3fb5fde458474d91b7b4f26d690354 2023-03-23 08:55:14,039 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del 2023-03-23 08:55:14,039 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del 2023-03-23 08:55:14,073 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc619e36f35949f48baeb572bff59300 2023-03-23 08:55:14,074 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/fc619e36f35949f48baeb572bff59300 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/fc619e36f35949f48baeb572bff59300 2023-03-23 08:55:14,084 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fc619e36f35949f48baeb572bff59300 2023-03-23 08:55:14,085 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in caf47a88030410666d2a8cb9e80797cb/family1 of caf47a88030410666d2a8cb9e80797cb into fc619e36f35949f48baeb572bff59300(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:14,085 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:14,085 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:14,086 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): caf47a88030410666d2a8cb9e80797cb/family2 is initiating major compaction (all files) 2023-03-23 08:55:14,086 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:14,086 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:14,087 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of caf47a88030410666d2a8cb9e80797cb/family2 in testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:14,087 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp, totalSize=25.4 K 2023-03-23 08:55:14,087 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting a406ffd2aeee4b648dff9d910f16beea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:14,088 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 287701c665e84e268ec9aa86a1fc086b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:14,088 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 708212eed5214c2382a1633ffb2a1f42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:14,089 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting f96299eaba8b4f74b1bcd32b38950406, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:14,089 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 31b9267f3aa9465a9c8e6098bd74e7d1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-23 08:55:14,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50] to archive 2023-03-23 08:55:14,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:14,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/4466553c64d749998af888c77372563b 2023-03-23 08:55:14,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/a0e51d5671ae42de98d6b5c0a0580d37 2023-03-23 08:55:14,126 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): caf47a88030410666d2a8cb9e80797cb#family2#compaction#113 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:14,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/033a0585ec65432280f909c048b2a09c 2023-03-23 08:55:14,131 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/317ffa2f61ee4360963349cc7a6188d9 2023-03-23 08:55:14,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/ad8dd7719c42421491817d351f3d2a50 2023-03-23 08:55:14,562 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del 2023-03-23 08:55:14,563 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del 2023-03-23 08:55:14,995 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fca07ec8d1a24e8bb418dedab68ad332 2023-03-23 08:55:14,998 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/fca07ec8d1a24e8bb418dedab68ad332 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fca07ec8d1a24e8bb418dedab68ad332 2023-03-23 08:55:15,009 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fca07ec8d1a24e8bb418dedab68ad332 2023-03-23 08:55:15,011 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in caf47a88030410666d2a8cb9e80797cb/family2 of caf47a88030410666d2a8cb9e80797cb into fca07ec8d1a24e8bb418dedab68ad332(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:15,012 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:15,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1] to archive 2023-03-23 08:55:15,027 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:15,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/a406ffd2aeee4b648dff9d910f16beea 2023-03-23 08:55:15,031 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/287701c665e84e268ec9aa86a1fc086b 2023-03-23 08:55:15,031 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:15,036 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/708212eed5214c2382a1633ffb2a1f42 2023-03-23 08:55:15,038 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/f96299eaba8b4f74b1bcd32b38950406 2023-03-23 08:55:15,040 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:15,040 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/31b9267f3aa9465a9c8e6098bd74e7d1 2023-03-23 08:55:15,056 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:15,060 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:15,084 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:55:15,140 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-23 08:55:15,144 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del 2023-03-23 08:55:15,147 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del 2023-03-23 08:55:15,153 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del 2023-03-23 08:55:15,153 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:15,153 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(139): Connect 0x45c0d03f to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:15,177 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3bdfe808, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:15,177 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:15,177 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:15,187 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:15,794 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd 2023-03-23 08:55:15,795 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:15,811 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:15,823 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40684, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:15,831 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:55:15,855 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49986, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:55:15,873 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/07261999fb3e4e82ac38a898917c8be5 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-23 08:55:15,874 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/07261999fb3e4e82ac38a898917c8be5}] 2023-03-23 08:55:15,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/07261999fb3e4e82ac38a898917c8be5 for inclusion in caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:15,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-23 08:55:15,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-23 08:55:15,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:15,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/07261999fb3e4e82ac38a898917c8be5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__otoid0usbqktm130905kpe6atg0s3dn8ova87chn9vladhgtaa0u7ikccrci2dih/family1/07261999fb3e4e82ac38a898917c8be5 2023-03-23 08:55:15,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__otoid0usbqktm130905kpe6atg0s3dn8ova87chn9vladhgtaa0u7ikccrci2dih/family1/07261999fb3e4e82ac38a898917c8be5 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ 2023-03-23 08:55:15,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__otoid0usbqktm130905kpe6atg0s3dn8ova87chn9vladhgtaa0u7ikccrci2dih/family1/07261999fb3e4e82ac38a898917c8be5 into caf47a88030410666d2a8cb9e80797cb/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ - updating store file list. 2023-03-23 08:55:15,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ into caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:15,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__otoid0usbqktm130905kpe6atg0s3dn8ova87chn9vladhgtaa0u7ikccrci2dih/family1/07261999fb3e4e82ac38a898917c8be5 into caf47a88030410666d2a8cb9e80797cb/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_) 2023-03-23 08:55:15,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__otoid0usbqktm130905kpe6atg0s3dn8ova87chn9vladhgtaa0u7ikccrci2dih/family1/07261999fb3e4e82ac38a898917c8be5 2023-03-23 08:55:15,901 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:15,901 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:15,903 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 2023-03-23 08:55:15,904 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 2023-03-23 08:55:15,905 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 2023-03-23 08:55:15,906 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f 2023-03-23 08:55:15,906 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:15,906 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323' 2023-03-23 08:55:15,906 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:16,178 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f 2023-03-23 08:55:16,179 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:16,195 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef 2023-03-23 08:55:16,196 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:16,215 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/ec26cde8d29f4c6288c87a05133631ec first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-23 08:55:16,218 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/ec26cde8d29f4c6288c87a05133631ec}] 2023-03-23 08:55:16,220 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/ec26cde8d29f4c6288c87a05133631ec for inclusion in 1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:16,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-23 08:55:16,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-23 08:55:16,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:16,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/ec26cde8d29f4c6288c87a05133631ec to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__cp56lr25ohmmljt6q54u0hbcn41om4296mtqvb049gu130aa36m7avlrqe0422q8/family1/ec26cde8d29f4c6288c87a05133631ec 2023-03-23 08:55:16,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__cp56lr25ohmmljt6q54u0hbcn41om4296mtqvb049gu130aa36m7avlrqe0422q8/family1/ec26cde8d29f4c6288c87a05133631ec as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ 2023-03-23 08:55:16,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__cp56lr25ohmmljt6q54u0hbcn41om4296mtqvb049gu130aa36m7avlrqe0422q8/family1/ec26cde8d29f4c6288c87a05133631ec into 1da1b8621f257956854515270c9db5ce/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ - updating store file list. 2023-03-23 08:55:16,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ into 1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:16,233 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__cp56lr25ohmmljt6q54u0hbcn41om4296mtqvb049gu130aa36m7avlrqe0422q8/family1/ec26cde8d29f4c6288c87a05133631ec into 1da1b8621f257956854515270c9db5ce/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_) 2023-03-23 08:55:16,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__cp56lr25ohmmljt6q54u0hbcn41om4296mtqvb049gu130aa36m7avlrqe0422q8/family1/ec26cde8d29f4c6288c87a05133631ec 2023-03-23 08:55:16,236 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:16,237 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:16,241 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 2023-03-23 08:55:16,243 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 2023-03-23 08:55:16,243 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/607d59249ba34e4fbbd8c8efd6b91f57 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-23 08:55:16,244 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/607d59249ba34e4fbbd8c8efd6b91f57}] 2023-03-23 08:55:16,244 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 2023-03-23 08:55:16,245 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e 2023-03-23 08:55:16,245 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:16,246 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323' 2023-03-23 08:55:16,246 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:16,246 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/607d59249ba34e4fbbd8c8efd6b91f57 for inclusion in 6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:16,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-23 08:55:16,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last=1 2023-03-23 08:55:16,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:16,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/607d59249ba34e4fbbd8c8efd6b91f57 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__tjlo5qs1sj10t8kik3hsvbs1gm5kn1k0ud4aj3ibi3nrrfri3mnhfcge60gi505k/family1/607d59249ba34e4fbbd8c8efd6b91f57 2023-03-23 08:55:16,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__tjlo5qs1sj10t8kik3hsvbs1gm5kn1k0ud4aj3ibi3nrrfri3mnhfcge60gi505k/family1/607d59249ba34e4fbbd8c8efd6b91f57 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ 2023-03-23 08:55:16,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__tjlo5qs1sj10t8kik3hsvbs1gm5kn1k0ud4aj3ibi3nrrfri3mnhfcge60gi505k/family1/607d59249ba34e4fbbd8c8efd6b91f57 into 6469db6845c8da10a08493a71e41b196/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ - updating store file list. 2023-03-23 08:55:16,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ into 6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:16,256 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__tjlo5qs1sj10t8kik3hsvbs1gm5kn1k0ud4aj3ibi3nrrfri3mnhfcge60gi505k/family1/607d59249ba34e4fbbd8c8efd6b91f57 into 6469db6845c8da10a08493a71e41b196/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_) 2023-03-23 08:55:16,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__tjlo5qs1sj10t8kik3hsvbs1gm5kn1k0ud4aj3ibi3nrrfri3mnhfcge60gi505k/family1/607d59249ba34e4fbbd8c8efd6b91f57 2023-03-23 08:55:16,264 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:16,265 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:16,267 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a 2023-03-23 08:55:16,269 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 2023-03-23 08:55:16,270 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c 2023-03-23 08:55:16,271 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a 2023-03-23 08:55:16,271 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:16,271 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323' 2023-03-23 08:55:16,271 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:16,271 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:16,271 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:16,272 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del] 2023-03-23 08:55:16,272 DEBUG [Listener at localhost/38371] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:16,273 DEBUG [Listener at localhost/38371] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e9571581d96d423f89659fd6d50e2c3c_del 2023-03-23 08:55:16,274 INFO [Listener at localhost/38371] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-23 08:55:16,274 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del] 2023-03-23 08:55:16,274 DEBUG [Listener at localhost/38371] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:16,275 DEBUG [Listener at localhost/38371] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032320f53b7bf91e4c9e9eb4280d99ccce08_del 2023-03-23 08:55:16,275 INFO [Listener at localhost/38371] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-23 08:55:16,275 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del] 2023-03-23 08:55:16,276 DEBUG [Listener at localhost/38371] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:16,277 DEBUG [Listener at localhost/38371] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230323264b1a5cc5fd4857877153ff6e55749b_del 2023-03-23 08:55:16,277 INFO [Listener at localhost/38371] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-23 08:55:16,280 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:16,283 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:16,286 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:16,287 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:16,294 INFO [Listener at localhost/38371] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-23 08:55:16,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-23 08:55:16,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-23 08:55:16,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-23 08:55:16,298 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561716298"}]},"ts":"1679561716298"} 2023-03-23 08:55:16,300 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-23 08:55:16,301 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-23 08:55:16,302 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, UNASSIGN}] 2023-03-23 08:55:16,303 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, UNASSIGN 2023-03-23 08:55:16,303 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, UNASSIGN 2023-03-23 08:55:16,303 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, UNASSIGN 2023-03-23 08:55:16,304 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:16,304 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:16,304 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561716304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561716304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561716304"}]},"ts":"1679561716304"} 2023-03-23 08:55:16,304 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:16,304 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561716304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561716304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561716304"}]},"ts":"1679561716304"} 2023-03-23 08:55:16,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561716304"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561716304"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561716304"}]},"ts":"1679561716304"} 2023-03-23 08:55:16,305 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:16,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:16,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:16,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-23 08:55:16,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:16,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6469db6845c8da10a08493a71e41b196, disabling compactions & flushes 2023-03-23 08:55:16,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:16,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:16,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. after waiting 0 ms 2023-03-23 08:55:16,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:16,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-23 08:55:16,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:16,469 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:16,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:16,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:16,471 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=CLOSED 2023-03-23 08:55:16,471 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing caf47a88030410666d2a8cb9e80797cb, disabling compactions & flushes 2023-03-23 08:55:16,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:16,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:16,472 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561716471"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561716471"}]},"ts":"1679561716471"} 2023-03-23 08:55:16,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. after waiting 0 ms 2023-03-23 08:55:16,472 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:16,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-03-23 08:55:16,477 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941 in 168 msec 2023-03-23 08:55:16,479 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, UNASSIGN in 175 msec 2023-03-23 08:55:16,479 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-23 08:55:16,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:16,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:16,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:16,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:16,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1da1b8621f257956854515270c9db5ce, disabling compactions & flushes 2023-03-23 08:55:16,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:16,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:16,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. after waiting 0 ms 2023-03-23 08:55:16,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:16,482 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=CLOSED 2023-03-23 08:55:16,482 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561716482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561716482"}]},"ts":"1679561716482"} 2023-03-23 08:55:16,485 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-03-23 08:55:16,485 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941 in 178 msec 2023-03-23 08:55:16,486 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, UNASSIGN in 183 msec 2023-03-23 08:55:16,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-23 08:55:16,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:16,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:16,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:16,492 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=CLOSED 2023-03-23 08:55:16,492 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561716492"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561716492"}]},"ts":"1679561716492"} 2023-03-23 08:55:16,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-23 08:55:16,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941 in 189 msec 2023-03-23 08:55:16,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=88 2023-03-23 08:55:16,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, UNASSIGN in 194 msec 2023-03-23 08:55:16,498 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561716498"}]},"ts":"1679561716498"} 2023-03-23 08:55:16,500 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-23 08:55:16,501 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-23 08:55:16,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 207 msec 2023-03-23 08:55:16,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-23 08:55:16,601 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-23 08:55:16,612 INFO [Listener at localhost/38371] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:16,613 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:16,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679561716613 (current time:1679561716613). 2023-03-23 08:55:16,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-23 08:55:16,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] zookeeper.ReadOnlyZKClient(139): Connect 0x485949c5 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:16,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65b3f00a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:16,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:16,630 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40690, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:16,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x485949c5 to 127.0.0.1:50031 2023-03-23 08:55:16,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:16,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-23 08:55:16,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:16,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:16,635 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:16,636 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679562316636 2023-03-23 08:55:16,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:16,636 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1679561704545-1679561716612 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-23 08:55:16,637 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:16,637 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-23 08:55:16,737 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:16,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:16,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:16,738 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-23 08:55:16,938 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:16,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:16,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:16,940 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-23 08:55:17,043 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-23 08:55:17,045 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-23 08:55:17,060 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef] hfiles 2023-03-23 08:55:17,060 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd 2023-03-23 08:55:17,061 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f 2023-03-23 08:55:17,061 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef 2023-03-23 08:55:17,061 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/98d295c566a44f67a1326ceed6ce82b7, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_] hfiles 2023-03-23 08:55:17,061 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/98d295c566a44f67a1326ceed6ce82b7 2023-03-23 08:55:17,061 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ 2023-03-23 08:55:17,063 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/fc619e36f35949f48baeb572bff59300] hfiles 2023-03-23 08:55:17,064 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ 2023-03-23 08:55:17,064 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/fc619e36f35949f48baeb572bff59300 2023-03-23 08:55:17,064 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/a5bb573a08514599a6decb4b7a18ae9e] hfiles 2023-03-23 08:55:17,064 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ 2023-03-23 08:55:17,064 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/a5bb573a08514599a6decb4b7a18ae9e 2023-03-23 08:55:17,072 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/bca642edc6d74b5db4c91ffa16f1769c] hfiles 2023-03-23 08:55:17,073 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/80f9a9af932f4461b3e5ea16a1098cb3] hfiles 2023-03-23 08:55:17,073 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/bca642edc6d74b5db4c91ffa16f1769c 2023-03-23 08:55:17,073 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fca07ec8d1a24e8bb418dedab68ad332] hfiles 2023-03-23 08:55:17,073 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fca07ec8d1a24e8bb418dedab68ad332 2023-03-23 08:55:17,073 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/80f9a9af932f4461b3e5ea16a1098cb3 2023-03-23 08:55:17,086 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696] hfiles 2023-03-23 08:55:17,086 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del 2023-03-23 08:55:17,086 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303233d297da68fd44fc9a79c70cd23261d72 2023-03-23 08:55:17,086 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032379216fef2828440fb9d3f6938b4bf63d 2023-03-23 08:55:17,086 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303239b42ed13f75945919a0f6ae742237fcd 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230323bab96201e36440a5bc58e88c568b0c42 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323853c7dff7b224bb8b864fff0d0dc463d 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303239e0833fd9baa48c9b8a1b37cd2c3d9e0 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac5ad11ed3514624ad7e9f95770e404d 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230323f8345e303f0749f5b4eb3418529334fc 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032330f4c0ce8c6b46b1974b4a26da73e088 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303238eddda653b0b451597124f220072de17 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323ddc96a65a0f74ab3ba286855f9f0780d 2023-03-23 08:55:17,087 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323fda3d44596684088acf56c5435c30696 2023-03-23 08:55:17,105 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-23 08:55:17,105 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:17,106 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:17,240 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:17,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:17,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:17,242 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-23 08:55:17,438 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-23 08:55:17,440 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:17,440 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45c0d03f to 127.0.0.1:50031 2023-03-23 08:55:17,440 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:17,441 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:17,441 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33e43777 to 127.0.0.1:50031 2023-03-23 08:55:17,441 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:17,441 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:17,441 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4a2265bd to 127.0.0.1:50031 2023-03-23 08:55:17,441 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:17,442 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:17,442 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d28c898 to 127.0.0.1:50031 2023-03-23 08:55:17,442 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:17,442 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x56f46416 to 127.0.0.1:50031 2023-03-23 08:55:17,442 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:17,539 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679561704545-1679561716612 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:17,742 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:17,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:17,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-23 08:55:17,743 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-23 08:55:17,962 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1679561704545-1679561716612 of table testCompactionWithHFileLink completed 2023-03-23 08:55:17,962 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:17,962 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:17,962 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1679561704545-1679561716612 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679561716636Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1679561717043 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1679561704545-1679561716612 at 1679561717105 (+62 ms)Loading Region manifests for hbase-failsafe-snaptb-1679561704545-1679561716612 at 1679561717105Writing data manifest for hbase-failsafe-snaptb-1679561704545-1679561716612 at 1679561717109 (+4 ms)Verifying snapshot: hbase-failsafe-snaptb-1679561704545-1679561716612 at 1679561717518 (+409 ms)Snapshot hbase-failsafe-snaptb-1679561704545-1679561716612 of table testCompactionWithHFileLink completed at 1679561717962 (+444 ms) 2023-03-23 08:55:17,963 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-23 08:55:17,965 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.3300 sec 2023-03-23 08:55:18,744 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-23 08:55:18,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-23 08:55:18,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1679561704545-1679561716612 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-23 08:55:18,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679561704545" table: "testCompactionWithHFileLink" creation_time: 1679561704554 type: FLUSH version: 2 ttl: 0 ) 2023-03-23 08:55:18,799 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1679561704545 as table=testCompactionWithHFileLink 2023-03-23 08:55:18,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-23 08:55:18,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-23 08:55:19,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-23 08:55:19,210 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-23 08:55:19,211 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-23 08:55:19,211 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-23 08:55:19,212 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-23 08:55:19,217 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1679561704545" table: "testCompactionWithHFileLink" creation_time: 1679561704554 type: FLUSH version: 2 ttl: 0 2023-03-23 08:55:19,218 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink 2023-03-23 08:55:19,223 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-23 08:55:19,226 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323dbee7d0692c94f0f939413ef9cb1f9fd 2023-03-23 08:55:19,228 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032303a87b9ea23748bbaf70555f6d56e57f 2023-03-23 08:55:19,229 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230323e25958251953430d8403ab45e82488ef 2023-03-23 08:55:19,229 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,232 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,235 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,239 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,242 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,244 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,252 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,255 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,259 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,269 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,274 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,283 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a of snapshot=snaptb-1679561704545 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-23 08:55:19,295 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230323c573038a1bc24eb6b20352f48b7f6726_del 2023-03-23 08:55:19,297 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303232982b9c458f94109aa89593bfa15e916_del 2023-03-23 08:55:19,299 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032366f64ab8028041b0a927bf4e5e9b1d2a_del 2023-03-23 08:55:19,299 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,299 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:19,299 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(223): region to restore: caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:19,303 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/a5bb573a08514599a6decb4b7a18ae9e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/a5bb573a08514599a6decb4b7a18ae9e 2023-03-23 08:55:19,303 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/98d295c566a44f67a1326ceed6ce82b7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/98d295c566a44f67a1326ceed6ce82b7 2023-03-23 08:55:19,304 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/21574a3b173e485a98fb848168a4382f_SeqId_29_ 2023-03-23 08:55:19,305 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/1e562393ac514053bf34bcbf8507986f_SeqId_29_ 2023-03-23 08:55:19,305 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c7580ba1bc7d4778a981d28b8c652243 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,305 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/dc17175c294a422aa46f3e439a7e029e_SeqId_29_ 2023-03-23 08:55:19,305 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1310ee368980452198ba3bbb20ce6aa3 of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,306 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/fc619e36f35949f48baeb572bff59300 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/fc619e36f35949f48baeb572bff59300 2023-03-23 08:55:19,306 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4466553c64d749998af888c77372563b of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,313 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2caf681daca6482a8a91e6719e7f2d54 of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,314 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f3c0d0d20baa4ea7bab197f5a521cb04 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,314 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a0e51d5671ae42de98d6b5c0a0580d37 of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,317 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1cda00ec17d5478c843ba62cb80f15bb of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,318 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 10d486d59b3d4ab981afd70ee16f96c2 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,320 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 033a0585ec65432280f909c048b2a09c of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,321 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 14a964415ddb487aa266f359e7a82522 of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,326 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 317ffa2f61ee4360963349cc7a6188d9 of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,327 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5d7c3fa6815248dfbc446d05d5dfb8f2 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,327 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/bca642edc6d74b5db4c91ffa16f1769c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/bca642edc6d74b5db4c91ffa16f1769c 2023-03-23 08:55:19,327 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6eac470c41cd48cd836ec110e27b664f of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,339 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7f22b77db3614734af37d10795d0f832 of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,340 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fca07ec8d1a24e8bb418dedab68ad332 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fca07ec8d1a24e8bb418dedab68ad332 2023-03-23 08:55:19,340 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a406ffd2aeee4b648dff9d910f16beea of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,353 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 287701c665e84e268ec9aa86a1fc086b of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,353 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2182fa1a0d724cdcbe69be3b24d5aaa9 of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,354 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/80f9a9af932f4461b3e5ea16a1098cb3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/80f9a9af932f4461b3e5ea16a1098cb3 2023-03-23 08:55:19,354 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0d2ffc4d5cfa4e538a8b3f96c2647908 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,357 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 708212eed5214c2382a1633ffb2a1f42 of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,359 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink eb4ceef45c04496c925124fb07d5e86d of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,359 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink fcc0807b1af94e7aaefe81a62253caeb of snapshot=snaptb-1679561704545 to region=6469db6845c8da10a08493a71e41b196 table=testCompactionWithHFileLink 2023-03-23 08:55:19,364 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f96299eaba8b4f74b1bcd32b38950406 of snapshot=snaptb-1679561704545 to region=caf47a88030410666d2a8cb9e80797cb table=testCompactionWithHFileLink 2023-03-23 08:55:19,364 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f4296c5012e64ac8b5e771cad757469f of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,370 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e2cb4c734e224bce9e4c8cc16e3e31c4 of snapshot=snaptb-1679561704545 to region=1da1b8621f257956854515270c9db5ce table=testCompactionWithHFileLink 2023-03-23 08:55:19,374 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1679561704545" table: "testCompactionWithHFileLink" creation_time: 1679561704554 type: FLUSH version: 2 ttl: 0 2023-03-23 08:55:19,376 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679561719376"}]},"ts":"9223372036854775807"} 2023-03-23 08:55:19,376 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679561719376"}]},"ts":"9223372036854775807"} 2023-03-23 08:55:19,376 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679561719376"}]},"ts":"9223372036854775807"} 2023-03-23 08:55:19,378 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-23 08:55:19,378 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1da1b8621f257956854515270c9db5ce, NAME => 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6469db6845c8da10a08493a71e41b196, NAME => 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => caf47a88030410666d2a8cb9e80797cb, NAME => 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-23 08:55:19,379 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561719377"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561719377"}]},"ts":"1679561719377"} 2023-03-23 08:55:19,379 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561719377"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561719377"}]},"ts":"1679561719377"} 2023-03-23 08:55:19,379 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561719377"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561719377"}]},"ts":"1679561719377"} 2023-03-23 08:55:19,381 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-23 08:55:19,381 INFO [PEWorker-5] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-23 08:55:19,381 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 1da1b8621f257956854515270c9db5ce, NAME => 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 6469db6845c8da10a08493a71e41b196, NAME => 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => caf47a88030410666d2a8cb9e80797cb, NAME => 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-23 08:55:19,381 INFO [PEWorker-5] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1679561704545 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-23 08:55:19,383 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679561704545" table: "testCompactionWithHFileLink" creation_time: 1679561704554 type: FLUSH version: 2 ttl: 0 ) in 592 msec 2023-03-23 08:55:19,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-23 08:55:19,407 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-23 08:55:19,408 INFO [Listener at localhost/38371] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:19,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1679561704545-1679561716612" 2023-03-23 08:55:19,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1679561704545-1679561716612 2023-03-23 08:55:19,422 INFO [Listener at localhost/38371] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-23 08:55:19,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-23 08:55:19,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-23 08:55:19,440 INFO [PEWorker-2] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-23 08:55:19,440 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561719440"}]},"ts":"1679561719440"} 2023-03-23 08:55:19,442 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-23 08:55:19,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-23 08:55:19,445 DEBUG [PEWorker-2] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-23 08:55:19,446 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN}] 2023-03-23 08:55:19,447 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN 2023-03-23 08:55:19,448 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN 2023-03-23 08:55:19,448 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN 2023-03-23 08:55:19,449 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-23 08:55:19,449 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-23 08:55:19,449 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-23 08:55:19,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-23 08:55:19,601 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:19,601 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:19,601 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561719601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561719601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561719601"}]},"ts":"1679561719601"} 2023-03-23 08:55:19,601 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:19,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561719601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561719601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561719601"}]},"ts":"1679561719601"} 2023-03-23 08:55:19,601 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561719601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561719601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561719601"}]},"ts":"1679561719601"} 2023-03-23 08:55:19,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:19,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:19,604 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:19,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-23 08:55:19,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:19,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1da1b8621f257956854515270c9db5ce, NAME => 'testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.', STARTKEY => '2', ENDKEY => ''} 2023-03-23 08:55:19,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:19,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,759 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,760 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:19,760 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:19,760 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1da1b8621f257956854515270c9db5ce columnFamilyName family1 2023-03-23 08:55:19,774 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2 2023-03-23 08:55:19,780 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2 2023-03-23 08:55:19,792 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243 2023-03-23 08:55:19,798 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04 2023-03-23 08:55:19,798 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(310): Store=1da1b8621f257956854515270c9db5ce/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:19,798 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,799 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2 2023-03-23 08:55:19,799 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2 2023-03-23 08:55:19,800 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1da1b8621f257956854515270c9db5ce columnFamilyName family2 2023-03-23 08:55:19,816 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908 2023-03-23 08:55:19,830 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4 2023-03-23 08:55:19,839 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d 2023-03-23 08:55:19,850 DEBUG [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f 2023-03-23 08:55:19,850 INFO [StoreOpener-1da1b8621f257956854515270c9db5ce-1] regionserver.HStore(310): Store=1da1b8621f257956854515270c9db5ce/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:19,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:19,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:55:19,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1da1b8621f257956854515270c9db5ce; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73986056, jitterRate=0.10247814655303955}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:19,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:19,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., pid=101, masterSystemTime=1679561719754 2023-03-23 08:55:19,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-23 08:55:19,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-23 08:55:19,862 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-23 08:55:19,862 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-23 08:55:19,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:19,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:19,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:19,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6469db6845c8da10a08493a71e41b196, NAME => 'testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.', STARTKEY => '', ENDKEY => '1'} 2023-03-23 08:55:19,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:19,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:19,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:19,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:19,866 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=1da1b8621f257956854515270c9db5ce, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:19,866 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561719866"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561719866"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561719866"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561719866"}]},"ts":"1679561719866"} 2023-03-23 08:55:19,877 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-03-23 08:55:19,877 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 1da1b8621f257956854515270c9db5ce, server=jenkins-hbase4.apache.org,32923,1679561616941 in 265 msec 2023-03-23 08:55:19,878 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=1da1b8621f257956854515270c9db5ce, ASSIGN in 431 msec 2023-03-23 08:55:19,878 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:19,878 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-23 08:55:19,878 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1912): 1da1b8621f257956854515270c9db5ce/family1 is initiating minor compaction (all files) 2023-03-23 08:55:19,879 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1da1b8621f257956854515270c9db5ce/family1 in testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:19,879 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-23 08:55:19,879 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp, totalSize=21.6 K 2023-03-23 08:55:19,879 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1912): 1da1b8621f257956854515270c9db5ce/family2 is initiating minor compaction (all files) 2023-03-23 08:55:19,879 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1da1b8621f257956854515270c9db5ce/family2 in testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:55:19,879 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp, totalSize=20.4 K 2023-03-23 08:55:19,880 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:19,880 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:19,880 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:19,881 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:19,881 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:19,881 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:19,882 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:19,881 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:19,882 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:19,883 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:19,887 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6469db6845c8da10a08493a71e41b196 columnFamilyName family1 2023-03-23 08:55:19,920 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3 2023-03-23 08:55:19,947 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522 2023-03-23 08:55:19,969 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb 2023-03-23 08:55:19,970 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1da1b8621f257956854515270c9db5ce#family1#compaction#115 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:19,972 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): 1da1b8621f257956854515270c9db5ce#family2#compaction#114 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:20,014 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54 2023-03-23 08:55:20,014 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(310): Store=6469db6845c8da10a08493a71e41b196/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:20,014 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:20,015 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2 2023-03-23 08:55:20,016 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2 2023-03-23 08:55:20,016 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6469db6845c8da10a08493a71e41b196 columnFamilyName family2 2023-03-23 08:55:20,036 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9 2023-03-23 08:55:20,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-23 08:55:20,048 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f 2023-03-23 08:55:20,054 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832 2023-03-23 08:55:20,062 DEBUG [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb 2023-03-23 08:55:20,062 INFO [StoreOpener-6469db6845c8da10a08493a71e41b196-1] regionserver.HStore(310): Store=6469db6845c8da10a08493a71e41b196/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:20,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:20,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:20,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:55:20,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6469db6845c8da10a08493a71e41b196; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64235398, jitterRate=-0.04281798005104065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:20,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:20,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., pid=102, masterSystemTime=1679561719754 2023-03-23 08:55:20,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-23 08:55:20,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:20,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:20,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => caf47a88030410666d2a8cb9e80797cb, NAME => 'testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.', STARTKEY => '1', ENDKEY => '2'} 2023-03-23 08:55:20,071 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6469db6845c8da10a08493a71e41b196, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:20,071 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679561720071"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561720071"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561720071"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561720071"}]},"ts":"1679561720071"} 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,075 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-03-23 08:55:20,075 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 6469db6845c8da10a08493a71e41b196, server=jenkins-hbase4.apache.org,32923,1679561616941 in 470 msec 2023-03-23 08:55:20,076 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,076 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6469db6845c8da10a08493a71e41b196, ASSIGN in 629 msec 2023-03-23 08:55:20,077 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:20,077 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:20,077 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region caf47a88030410666d2a8cb9e80797cb columnFamilyName family1 2023-03-23 08:55:20,090 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c 2023-03-23 08:55:20,098 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9 2023-03-23 08:55:20,108 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b 2023-03-23 08:55:20,115 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37 2023-03-23 08:55:20,115 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(310): Store=caf47a88030410666d2a8cb9e80797cb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:20,116 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,117 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2 2023-03-23 08:55:20,117 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2 2023-03-23 08:55:20,117 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region caf47a88030410666d2a8cb9e80797cb columnFamilyName family2 2023-03-23 08:55:20,127 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b 2023-03-23 08:55:20,140 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42 2023-03-23 08:55:20,151 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea 2023-03-23 08:55:20,159 DEBUG [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(539): loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406 2023-03-23 08:55:20,159 INFO [StoreOpener-caf47a88030410666d2a8cb9e80797cb-1] regionserver.HStore(310): Store=caf47a88030410666d2a8cb9e80797cb/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:20,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:20,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:55:20,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened caf47a88030410666d2a8cb9e80797cb; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71804782, jitterRate=0.0699746310710907}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:20,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:20,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., pid=103, masterSystemTime=1679561719754 2023-03-23 08:55:20,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-23 08:55:20,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-23 08:55:20,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:20,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:20,169 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=caf47a88030410666d2a8cb9e80797cb, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:20,169 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561720168"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561720168"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561720168"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561720168"}]},"ts":"1679561720168"} 2023-03-23 08:55:20,173 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-03-23 08:55:20,173 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure caf47a88030410666d2a8cb9e80797cb, server=jenkins-hbase4.apache.org,32923,1679561616941 in 566 msec 2023-03-23 08:55:20,174 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-03-23 08:55:20,174 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=caf47a88030410666d2a8cb9e80797cb, ASSIGN in 727 msec 2023-03-23 08:55:20,176 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561720175"}]},"ts":"1679561720175"} 2023-03-23 08:55:20,177 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-23 08:55:20,178 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-23 08:55:20,179 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 749 msec 2023-03-23 08:55:20,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-23 08:55:20,547 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-23 08:55:20,561 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:20,571 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:20,581 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:20,582 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:20,588 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:55:20,606 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-23 08:55:20,606 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:20,607 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(139): Connect 0x1b29ae1b to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:20,611 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3577088e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:20,612 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:20,612 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:20,612 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:20,736 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032367b0b496ea71416bb3e878a5005d6e7b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032367b0b496ea71416bb3e878a5005d6e7b 2023-03-23 08:55:20,737 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:20,742 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032376696af3cec34c6fb078a9cc17f48f78 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032376696af3cec34c6fb078a9cc17f48f78 2023-03-23 08:55:20,742 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:20,755 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:20,761 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40694, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:20,771 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:55:20,787 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50002, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:55:20,817 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/6daf11f464c14af897e462f0936b636f first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-23 08:55:20,825 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/6daf11f464c14af897e462f0936b636f}] 2023-03-23 08:55:20,832 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/9ed4283cb7e446fea93cb89583bb7a7a first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-23 08:55:20,833 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family2/ba558f64a7084c17b9c0817686648ee4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/ba558f64a7084c17b9c0817686648ee4 2023-03-23 08:55:20,837 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1da1b8621f257956854515270c9db5ce/family2 of 1da1b8621f257956854515270c9db5ce into ba558f64a7084c17b9c0817686648ee4(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:20,837 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:20,837 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., storeName=1da1b8621f257956854515270c9db5ce/family2, priority=12, startTime=1679561719861; duration=0sec 2023-03-23 08:55:20,837 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-23 08:55:20,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4] to archive 2023-03-23 08:55:20,844 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-23 08:55:20,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:20,845 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=32 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/9ed4283cb7e446fea93cb89583bb7a7a}] 2023-03-23 08:55:20,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/6daf11f464c14af897e462f0936b636f for inclusion in 1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:20,850 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/9ed4283cb7e446fea93cb89583bb7a7a for inclusion in 6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:20,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-0d2ffc4d5cfa4e538a8b3f96c2647908 2023-03-23 08:55:20,851 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-23 08:55:20,851 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1912): 6469db6845c8da10a08493a71e41b196/family1 is initiating minor compaction (all files) 2023-03-23 08:55:20,851 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6469db6845c8da10a08493a71e41b196/family1 in testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:20,851 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp, totalSize=21.6 K 2023-03-23 08:55:20,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-eb4ceef45c04496c925124fb07d5e86d 2023-03-23 08:55:20,852 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698780 2023-03-23 08:55:20,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-23 08:55:20,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-23 08:55:20,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:20,854 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:20,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f4296c5012e64ac8b5e771cad757469f 2023-03-23 08:55:20,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323/family1/6daf11f464c14af897e462f0936b636f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__gdle9ghq18n61rfsa24equslg48enqigrn7dnbo83aveshhmcic38l0kd4gkrmcr/family1/6daf11f464c14af897e462f0936b636f 2023-03-23 08:55:20,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-23 08:55:20,855 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:20,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last=1 2023-03-23 08:55:20,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:20,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323/family1/9ed4283cb7e446fea93cb89583bb7a7a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__1ips93fhaic7b21efvr0ob7a1g0nrcaiif5je9mogg0a4tnkqiju0610b1mt8b6j/family1/9ed4283cb7e446fea93cb89583bb7a7a 2023-03-23 08:55:20,857 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:20,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family2/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-e2cb4c734e224bce9e4c8cc16e3e31c4 2023-03-23 08:55:20,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__1ips93fhaic7b21efvr0ob7a1g0nrcaiif5je9mogg0a4tnkqiju0610b1mt8b6j/family1/9ed4283cb7e446fea93cb89583bb7a7a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/95fc5186f54744d6aa21594377ab2151_SeqId_34_ 2023-03-23 08:55:20,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__1ips93fhaic7b21efvr0ob7a1g0nrcaiif5je9mogg0a4tnkqiju0610b1mt8b6j/family1/9ed4283cb7e446fea93cb89583bb7a7a into 6469db6845c8da10a08493a71e41b196/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/95fc5186f54744d6aa21594377ab2151_SeqId_34_ - updating store file list. 2023-03-23 08:55:20,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__gdle9ghq18n61rfsa24equslg48enqigrn7dnbo83aveshhmcic38l0kd4gkrmcr/family1/6daf11f464c14af897e462f0936b636f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/7cc9696178e9470486d6491f2a3a1e98_SeqId_35_ 2023-03-23 08:55:20,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__gdle9ghq18n61rfsa24equslg48enqigrn7dnbo83aveshhmcic38l0kd4gkrmcr/family1/6daf11f464c14af897e462f0936b636f into 1da1b8621f257956854515270c9db5ce/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/7cc9696178e9470486d6491f2a3a1e98_SeqId_35_ - updating store file list. 2023-03-23 08:55:20,870 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/95fc5186f54744d6aa21594377ab2151_SeqId_34_ into 6469db6845c8da10a08493a71e41b196/family1 2023-03-23 08:55:20,871 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__1ips93fhaic7b21efvr0ob7a1g0nrcaiif5je9mogg0a4tnkqiju0610b1mt8b6j/family1/9ed4283cb7e446fea93cb89583bb7a7a into 6469db6845c8da10a08493a71e41b196/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/95fc5186f54744d6aa21594377ab2151_SeqId_34_) 2023-03-23 08:55:20,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__1ips93fhaic7b21efvr0ob7a1g0nrcaiif5je9mogg0a4tnkqiju0610b1mt8b6j/family1/9ed4283cb7e446fea93cb89583bb7a7a 2023-03-23 08:55:20,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/7cc9696178e9470486d6491f2a3a1e98_SeqId_35_ into 1da1b8621f257956854515270c9db5ce/family1 2023-03-23 08:55:20,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__gdle9ghq18n61rfsa24equslg48enqigrn7dnbo83aveshhmcic38l0kd4gkrmcr/family1/6daf11f464c14af897e462f0936b636f into 1da1b8621f257956854515270c9db5ce/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/7cc9696178e9470486d6491f2a3a1e98_SeqId_35_) 2023-03-23 08:55:20,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__gdle9ghq18n61rfsa24equslg48enqigrn7dnbo83aveshhmcic38l0kd4gkrmcr/family1/6daf11f464c14af897e462f0936b636f 2023-03-23 08:55:20,896 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:20,896 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:20,897 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:20,897 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:20,898 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303233b9105a84db34fa3a4a7fcba9e2ee22a 2023-03-23 08:55:20,899 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032309946f075c0641e8ab827273ea823666 2023-03-23 08:55:20,900 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230323702735bbf88c4f7ba1744bdf208ca297 2023-03-23 08:55:20,900 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323385d124e3dca412d8393143d42d35434 2023-03-23 08:55:20,902 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303239224bb0fc107495ab8415dfa3970451c 2023-03-23 08:55:20,903 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323c31236cc88fb4f099ab54a8ad21ba4f6 2023-03-23 08:55:20,905 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230323e3b8db71f66f48aa834c69b7ef18f26a 2023-03-23 08:55:20,905 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:20,905 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230323c62504307e7d428a803f59620fdf452e 2023-03-23 08:55:20,906 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230323' 2023-03-23 08:55:20,906 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:20,906 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230323' 2023-03-23 08:55:20,906 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:20,906 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:20,916 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): 6469db6845c8da10a08493a71e41b196#family1#compaction#116 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:21,134 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323ec370487d85c4d22b2ab30d4f013825c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230323ec370487d85c4d22b2ab30d4f013825c 2023-03-23 08:55:21,135 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:21,161 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/6cb6b335888444709df5cc190cdd3b0a first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-23 08:55:21,162 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/6cb6b335888444709df5cc190cdd3b0a}] 2023-03-23 08:55:21,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/6cb6b335888444709df5cc190cdd3b0a for inclusion in caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:21,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-23 08:55:21,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-23 08:55:21,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:21,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323/family1/6cb6b335888444709df5cc190cdd3b0a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__t8hb46ncrbrfl6prd1ka19uk0sse5vh1i0up61rnlha8pd37o07df27l5taot6gr/family1/6cb6b335888444709df5cc190cdd3b0a 2023-03-23 08:55:21,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__t8hb46ncrbrfl6prd1ka19uk0sse5vh1i0up61rnlha8pd37o07df27l5taot6gr/family1/6cb6b335888444709df5cc190cdd3b0a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_ 2023-03-23 08:55:21,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__t8hb46ncrbrfl6prd1ka19uk0sse5vh1i0up61rnlha8pd37o07df27l5taot6gr/family1/6cb6b335888444709df5cc190cdd3b0a into caf47a88030410666d2a8cb9e80797cb/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_ - updating store file list. 2023-03-23 08:55:21,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_ into caf47a88030410666d2a8cb9e80797cb/family1 2023-03-23 08:55:21,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__t8hb46ncrbrfl6prd1ka19uk0sse5vh1i0up61rnlha8pd37o07df27l5taot6gr/family1/6cb6b335888444709df5cc190cdd3b0a into caf47a88030410666d2a8cb9e80797cb/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_) 2023-03-23 08:55:21,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testCompactionWithHFileLink__t8hb46ncrbrfl6prd1ka19uk0sse5vh1i0up61rnlha8pd37o07df27l5taot6gr/family1/6cb6b335888444709df5cc190cdd3b0a 2023-03-23 08:55:21,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:21,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-03-23 08:55:21,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): caf47a88030410666d2a8cb9e80797cb/family1 is initiating minor compaction (all files) 2023-03-23 08:55:21,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@14587cbb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-23 08:55:21,180 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:21,181 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:21,182 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032313a09e2aa7564244a623e3c8755febd9 2023-03-23 08:55:21,183 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303231a44ee2edd1f46cf834294ee3c534b45 2023-03-23 08:55:21,185 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230323cd6e28a9aed04fe99cf83eb3b70e9d32 2023-03-23 08:55:21,186 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230323dc07ac2082f345d5bdf105f0b28f887f 2023-03-23 08:55:21,186 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-23 08:55:21,186 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230323' 2023-03-23 08:55:21,186 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230323 for table='testCompactionWithHFileLink' and column='family1' 2023-03-23 08:55:21,186 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-23 08:55:21,189 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:21,191 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:21,195 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:21,196 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:21,206 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:21,207 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:21,223 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=284 (was 250) - Thread LEAK? -, OpenFileDescriptor=733 (was 633) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=371 (was 429), ProcessCount=169 (was 169), AvailableMemoryMB=5615 (was 5745) 2023-03-23 08:55:21,236 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=284, OpenFileDescriptor=733, MaxFileDescriptor=60000, SystemLoadAverage=371, ProcessCount=169, AvailableMemoryMB=5615 2023-03-23 08:55:21,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-23 08:55:21,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-23 08:55:21,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-23 08:55:21,248 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-23 08:55:21,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-03-23 08:55:21,253 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/.tmp/family1/9d14debb039a444ab4a00b8aa84a27c4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/9d14debb039a444ab4a00b8aa84a27c4 2023-03-23 08:55:21,257 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 1da1b8621f257956854515270c9db5ce/family1 of 1da1b8621f257956854515270c9db5ce into 9d14debb039a444ab4a00b8aa84a27c4(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:21,258 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:55:21,258 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce., storeName=1da1b8621f257956854515270c9db5ce/family1, priority=12, startTime=1679561719861; duration=1sec 2023-03-23 08:55:21,258 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-03-23 08:55:21,258 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of caf47a88030410666d2a8cb9e80797cb/family1 in testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:21,258 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp, totalSize=28.9 K 2023-03-23 08:55:21,258 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:21,259 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:21,260 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:21,260 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:21,260 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting c6eb06261f8c463898926a162b391ec4_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1679561698782 2023-03-23 08:55:21,275 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): caf47a88030410666d2a8cb9e80797cb#family1#compaction#117 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:21,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2] to archive 2023-03-23 08:55:21,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:21,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-c7580ba1bc7d4778a981d28b8c652243 2023-03-23 08:55:21,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-f3c0d0d20baa4ea7bab197f5a521cb04 2023-03-23 08:55:21,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-10d486d59b3d4ab981afd70ee16f96c2 2023-03-23 08:55:21,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/family1/testCompactionWithHFileLink=1da1b8621f257956854515270c9db5ce-5d7c3fa6815248dfbc446d05d5dfb8f2 2023-03-23 08:55:21,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-23 08:55:21,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-23 08:55:21,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-23 08:55:21,351 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:55:21,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-23 08:55:21,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-23 08:55:21,354 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:55:21,356 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:21,356 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:21,356 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:21,356 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af empty. 2023-03-23 08:55:21,358 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1 empty. 2023-03-23 08:55:21,358 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:21,358 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e empty. 2023-03-23 08:55:21,358 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:21,358 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:21,359 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-23 08:55:21,366 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family1/eea52f9203074bc6a491ccf9fdb4b669 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/eea52f9203074bc6a491ccf9fdb4b669 2023-03-23 08:55:21,372 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6469db6845c8da10a08493a71e41b196/family1 of 6469db6845c8da10a08493a71e41b196 into eea52f9203074bc6a491ccf9fdb4b669(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:21,372 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:21,372 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., storeName=6469db6845c8da10a08493a71e41b196/family1, priority=12, startTime=1679561720069; duration=0sec 2023-03-23 08:55:21,372 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-23 08:55:21,373 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-23 08:55:21,376 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-23 08:55:21,376 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1912): 6469db6845c8da10a08493a71e41b196/family2 is initiating minor compaction (all files) 2023-03-23 08:55:21,376 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6469db6845c8da10a08493a71e41b196/family2 in testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:55:21,376 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp, totalSize=20.4 K 2023-03-23 08:55:21,377 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698780 2023-03-23 08:55:21,378 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:21,378 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:21,379 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:21,392 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): 6469db6845c8da10a08493a71e41b196#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:21,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522] to archive 2023-03-23 08:55:21,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:21,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1310ee368980452198ba3bbb20ce6aa3 2023-03-23 08:55:21,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2caf681daca6482a8a91e6719e7f2d54 2023-03-23 08:55:21,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-23 08:55:21,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-1cda00ec17d5478c843ba62cb80f15bb 2023-03-23 08:55:21,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family1/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-14a964415ddb487aa266f359e7a82522 2023-03-23 08:55:21,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-23 08:55:21,779 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-23 08:55:21,781 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => d2a2668ed83ec24163021b10f82d1b3e, NAME => 'ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:21,781 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 8316b0845ad8dc2f32d05f84414a28c1, NAME => 'ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:21,781 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => d69f3c8983e2fde2ae01956e233272af, NAME => 'ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 8316b0845ad8dc2f32d05f84414a28c1, disabling compactions & flushes 2023-03-23 08:55:21,793 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. after waiting 0 ms 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:21,793 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing d2a2668ed83ec24163021b10f82d1b3e, disabling compactions & flushes 2023-03-23 08:55:21,793 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. after waiting 0 ms 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:21,793 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:21,793 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:21,795 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:21,795 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing d69f3c8983e2fde2ae01956e233272af, disabling compactions & flushes 2023-03-23 08:55:21,796 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:21,796 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:21,796 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. after waiting 0 ms 2023-03-23 08:55:21,796 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:21,796 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:21,796 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:21,798 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:55:21,800 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679561721799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561721799"}]},"ts":"1679561721799"} 2023-03-23 08:55:21,800 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561721799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561721799"}]},"ts":"1679561721799"} 2023-03-23 08:55:21,800 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561721799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561721799"}]},"ts":"1679561721799"} 2023-03-23 08:55:21,802 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-23 08:55:21,803 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:55:21,803 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561721803"}]},"ts":"1679561721803"} 2023-03-23 08:55:21,806 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-23 08:55:21,809 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d69f3c8983e2fde2ae01956e233272af, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8316b0845ad8dc2f32d05f84414a28c1, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2a2668ed83ec24163021b10f82d1b3e, ASSIGN}] 2023-03-23 08:55:21,821 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2a2668ed83ec24163021b10f82d1b3e, ASSIGN 2023-03-23 08:55:21,821 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8316b0845ad8dc2f32d05f84414a28c1, ASSIGN 2023-03-23 08:55:21,821 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d69f3c8983e2fde2ae01956e233272af, ASSIGN 2023-03-23 08:55:21,822 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2a2668ed83ec24163021b10f82d1b3e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:21,822 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8316b0845ad8dc2f32d05f84414a28c1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:21,822 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d69f3c8983e2fde2ae01956e233272af, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:21,832 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/.tmp/family2/99a7bdd3dfca4ee9b2dbe777f49ce23d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/99a7bdd3dfca4ee9b2dbe777f49ce23d 2023-03-23 08:55:21,836 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6469db6845c8da10a08493a71e41b196/family2 of 6469db6845c8da10a08493a71e41b196 into 99a7bdd3dfca4ee9b2dbe777f49ce23d(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:55:21,836 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196., storeName=6469db6845c8da10a08493a71e41b196/family2, priority=12, startTime=1679561720069; duration=0sec 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. because compaction request was cancelled 2023-03-23 08:55:21,836 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-23 08:55:21,838 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-23 08:55:21,839 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1912): caf47a88030410666d2a8cb9e80797cb/family2 is initiating minor compaction (all files) 2023-03-23 08:55:21,839 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2259): Starting compaction of caf47a88030410666d2a8cb9e80797cb/family2 in testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:55:21,839 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp, totalSize=20.4 K 2023-03-23 08:55:21,839 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561698782 2023-03-23 08:55:21,840 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561700233 2023-03-23 08:55:21,841 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561701736 2023-03-23 08:55:21,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb] to archive 2023-03-23 08:55:21,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:21,842 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561702740 2023-03-23 08:55:21,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-6eac470c41cd48cd836ec110e27b664f 2023-03-23 08:55:21,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-7f22b77db3614734af37d10795d0f832 2023-03-23 08:55:21,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-2182fa1a0d724cdcbe69be3b24d5aaa9 2023-03-23 08:55:21,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/family2/testCompactionWithHFileLink=6469db6845c8da10a08493a71e41b196-fcc0807b1af94e7aaefe81a62253caeb 2023-03-23 08:55:21,857 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] throttle.PressureAwareThroughputController(145): caf47a88030410666d2a8cb9e80797cb#family2#compaction#119 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:21,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-23 08:55:21,975 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d2a2668ed83ec24163021b10f82d1b3e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:21,975 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d69f3c8983e2fde2ae01956e233272af, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:21,975 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=8316b0845ad8dc2f32d05f84414a28c1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:21,975 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561721975"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561721975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561721975"}]},"ts":"1679561721975"} 2023-03-23 08:55:21,975 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561721975"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561721975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561721975"}]},"ts":"1679561721975"} 2023-03-23 08:55:21,976 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679561721975"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561721975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561721975"}]},"ts":"1679561721975"} 2023-03-23 08:55:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure d2a2668ed83ec24163021b10f82d1b3e, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:21,978 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure d69f3c8983e2fde2ae01956e233272af, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:21,978 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=107, state=RUNNABLE; OpenRegionProcedure 8316b0845ad8dc2f32d05f84414a28c1, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:22,029 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:22,103 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family1/c1500082f27949cfab404ceb4650cfa7 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c1500082f27949cfab404ceb4650cfa7 2023-03-23 08:55:22,108 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in caf47a88030410666d2a8cb9e80797cb/family1 of caf47a88030410666d2a8cb9e80797cb into c1500082f27949cfab404ceb4650cfa7(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:22,108 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:22,109 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1679561721177; duration=0sec 2023-03-23 08:55:22,109 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:22,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:22,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8316b0845ad8dc2f32d05f84414a28c1, NAME => 'ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-23 08:55:22,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:22,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,136 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,138 DEBUG [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1 2023-03-23 08:55:22,138 DEBUG [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1 2023-03-23 08:55:22,138 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8316b0845ad8dc2f32d05f84414a28c1 columnFamilyName family1 2023-03-23 08:55:22,140 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] regionserver.HStore(310): Store=8316b0845ad8dc2f32d05f84414a28c1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,140 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,141 DEBUG [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2 2023-03-23 08:55:22,141 DEBUG [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2 2023-03-23 08:55:22,142 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8316b0845ad8dc2f32d05f84414a28c1 columnFamilyName family2 2023-03-23 08:55:22,143 INFO [StoreOpener-8316b0845ad8dc2f32d05f84414a28c1-1] regionserver.HStore(310): Store=8316b0845ad8dc2f32d05f84414a28c1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_] to archive 2023-03-23 08:55:22,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:22,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-4466553c64d749998af888c77372563b 2023-03-23 08:55:22,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:22,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:55:22,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a0e51d5671ae42de98d6b5c0a0580d37 2023-03-23 08:55:22,163 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-033a0585ec65432280f909c048b2a09c 2023-03-23 08:55:22,164 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-317ffa2f61ee4360963349cc7a6188d9 2023-03-23 08:55:22,165 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family1/c6eb06261f8c463898926a162b391ec4_SeqId_34_ 2023-03-23 08:55:22,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:22,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8316b0845ad8dc2f32d05f84414a28c1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71174640, jitterRate=0.06058478355407715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:22,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:22,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1., pid=111, masterSystemTime=1679561722130 2023-03-23 08:55:22,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:22,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:22,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:22,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2a2668ed83ec24163021b10f82d1b3e, NAME => 'ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.', STARTKEY => '2', ENDKEY => ''} 2023-03-23 08:55:22,179 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=8316b0845ad8dc2f32d05f84414a28c1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:22,179 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679561722179"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561722179"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561722179"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561722179"}]},"ts":"1679561722179"} 2023-03-23 08:55:22,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:22,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,180 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,182 DEBUG [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1 2023-03-23 08:55:22,182 DEBUG [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1 2023-03-23 08:55:22,183 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2a2668ed83ec24163021b10f82d1b3e columnFamilyName family1 2023-03-23 08:55:22,184 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=107 2023-03-23 08:55:22,184 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=107, state=SUCCESS; OpenRegionProcedure 8316b0845ad8dc2f32d05f84414a28c1, server=jenkins-hbase4.apache.org,32923,1679561616941 in 202 msec 2023-03-23 08:55:22,184 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] regionserver.HStore(310): Store=d2a2668ed83ec24163021b10f82d1b3e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,185 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=8316b0845ad8dc2f32d05f84414a28c1, ASSIGN in 375 msec 2023-03-23 08:55:22,186 DEBUG [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2 2023-03-23 08:55:22,186 DEBUG [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2 2023-03-23 08:55:22,186 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2a2668ed83ec24163021b10f82d1b3e columnFamilyName family2 2023-03-23 08:55:22,187 INFO [StoreOpener-d2a2668ed83ec24163021b10f82d1b3e-1] regionserver.HStore(310): Store=d2a2668ed83ec24163021b10f82d1b3e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:22,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:55:22,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:22,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2a2668ed83ec24163021b10f82d1b3e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64650174, jitterRate=-0.036637336015701294}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:22,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:22,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e., pid=109, masterSystemTime=1679561722130 2023-03-23 08:55:22,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:22,196 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=d2a2668ed83ec24163021b10f82d1b3e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:22,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:22,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:22,196 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561722196"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561722196"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561722196"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561722196"}]},"ts":"1679561722196"} 2023-03-23 08:55:22,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d69f3c8983e2fde2ae01956e233272af, NAME => 'ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.', STARTKEY => '', ENDKEY => '1'} 2023-03-23 08:55:22,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:22,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,198 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-03-23 08:55:22,198 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure d2a2668ed83ec24163021b10f82d1b3e, server=jenkins-hbase4.apache.org,32923,1679561616941 in 219 msec 2023-03-23 08:55:22,199 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d2a2668ed83ec24163021b10f82d1b3e, ASSIGN in 389 msec 2023-03-23 08:55:22,200 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,202 DEBUG [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1 2023-03-23 08:55:22,202 DEBUG [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1 2023-03-23 08:55:22,202 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d69f3c8983e2fde2ae01956e233272af columnFamilyName family1 2023-03-23 08:55:22,203 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] regionserver.HStore(310): Store=d69f3c8983e2fde2ae01956e233272af/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,204 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,213 DEBUG [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2 2023-03-23 08:55:22,213 DEBUG [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2 2023-03-23 08:55:22,213 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d69f3c8983e2fde2ae01956e233272af columnFamilyName family2 2023-03-23 08:55:22,214 INFO [StoreOpener-d69f3c8983e2fde2ae01956e233272af-1] regionserver.HStore(310): Store=d69f3c8983e2fde2ae01956e233272af/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:22,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:22,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:55:22,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:22,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d69f3c8983e2fde2ae01956e233272af; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61493166, jitterRate=-0.0836804211139679}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:22,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:22,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af., pid=110, masterSystemTime=1679561722130 2023-03-23 08:55:22,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:22,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:22,227 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d69f3c8983e2fde2ae01956e233272af, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:22,227 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679561722227"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561722227"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561722227"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561722227"}]},"ts":"1679561722227"} 2023-03-23 08:55:22,230 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-03-23 08:55:22,230 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure d69f3c8983e2fde2ae01956e233272af, server=jenkins-hbase4.apache.org,32923,1679561616941 in 251 msec 2023-03-23 08:55:22,232 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-03-23 08:55:22,232 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=d69f3c8983e2fde2ae01956e233272af, ASSIGN in 421 msec 2023-03-23 08:55:22,232 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:55:22,232 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561722232"}]},"ts":"1679561722232"} 2023-03-23 08:55:22,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-23 08:55:22,238 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:55:22,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 889 msec 2023-03-23 08:55:22,309 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/.tmp/family2/fdc5e5cc018a4632b5943604bed6f940 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/fdc5e5cc018a4632b5943604bed6f940 2023-03-23 08:55:22,314 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in caf47a88030410666d2a8cb9e80797cb/family2 of caf47a88030410666d2a8cb9e80797cb into fdc5e5cc018a4632b5943604bed6f940(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:22,314 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:55:22,314 INFO [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb., storeName=caf47a88030410666d2a8cb9e80797cb/family2, priority=12, startTime=1679561720167; duration=0sec 2023-03-23 08:55:22,314 DEBUG [RS:0;jenkins-hbase4:32923-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:22,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406] to archive 2023-03-23 08:55:22,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:22,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-a406ffd2aeee4b648dff9d910f16beea 2023-03-23 08:55:22,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-287701c665e84e268ec9aa86a1fc086b 2023-03-23 08:55:22,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-708212eed5214c2382a1633ffb2a1f42 2023-03-23 08:55:22,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/family2/testCompactionWithHFileLink=caf47a88030410666d2a8cb9e80797cb-f96299eaba8b4f74b1bcd32b38950406 2023-03-23 08:55:22,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-23 08:55:22,457 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-23 08:55:22,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:22,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:22,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:22,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:22,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-23 08:55:22,493 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-23 08:55:22,493 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:22,493 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-23 08:55:22,493 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-23 08:55:22,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:22,494 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:22,496 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:22,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:22,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:22,496 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:22,496 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:22,497 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:22,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:22,497 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:22,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:22,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:22,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-23 08:55:22,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-23 08:55:22,499 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-23 08:55:22,499 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:22,499 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-23 08:55:22,500 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:22,500 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. started... 2023-03-23 08:55:22,504 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8316b0845ad8dc2f32d05f84414a28c1 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:22,504 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:22,505 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. started... 2023-03-23 08:55:22,508 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:22,517 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:22,517 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d69f3c8983e2fde2ae01956e233272af 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:22,519 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:22,519 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. started... 2023-03-23 08:55:22,521 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d2a2668ed83ec24163021b10f82d1b3e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:22,566 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303234606ec5377ba49e88a4aa044e36bdb94 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303234606ec5377ba49e88a4aa044e36bdb94 2023-03-23 08:55:22,579 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323ba594341367b46cc9b2de57c627e6cca to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323ba594341367b46cc9b2de57c627e6cca 2023-03-23 08:55:22,588 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/53b18717761e4d6fb5b51d697b903489 2023-03-23 08:55:22,966 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323d80f3265100845ad88ab17fe055549be to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230323d80f3265100845ad88ab17fe055549be 2023-03-23 08:55:22,982 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/30e89a86fae14b2ea22b0c846ee522b6 2023-03-23 08:55:23,011 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323df51ae26984d4428bc1e9c24d2d2f5d8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230323df51ae26984d4428bc1e9c24d2d2f5d8 2023-03-23 08:55:23,383 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/0cf7a9ba087e4a81ac2fd8718fc75768 2023-03-23 08:55:23,396 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303238893ec2889ad496ea89ac9544238c486 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303238893ec2889ad496ea89ac9544238c486 2023-03-23 08:55:23,406 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/ef6570714d3a4d418c21d82cba1926cb 2023-03-23 08:55:23,412 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303230ee19e00d10748dfa9304929d6609057 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303230ee19e00d10748dfa9304929d6609057 2023-03-23 08:55:23,413 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/30e89a86fae14b2ea22b0c846ee522b6 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6 2023-03-23 08:55:23,416 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/0113c8833f9349d895da7fb2387d6d18 2023-03-23 08:55:23,419 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:55:23,420 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/ef6570714d3a4d418c21d82cba1926cb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb 2023-03-23 08:55:23,421 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/53b18717761e4d6fb5b51d697b903489 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489 2023-03-23 08:55:23,423 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/4300e334fe734707a584895fe4736fd2 2023-03-23 08:55:23,425 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:23,426 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:55:23,426 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d69f3c8983e2fde2ae01956e233272af in 909ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:23,426 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-23 08:55:23,427 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:23,427 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:23,427 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:23,427 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/0113c8833f9349d895da7fb2387d6d18 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18 2023-03-23 08:55:23,428 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/0cf7a9ba087e4a81ac2fd8718fc75768 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768 2023-03-23 08:55:23,432 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:23,432 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8316b0845ad8dc2f32d05f84414a28c1 in 928ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:23,433 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:23,434 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:23,434 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:23,435 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768, entries=4, sequenceid=5, filesize=5.4 K 2023-03-23 08:55:23,436 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/4300e334fe734707a584895fe4736fd2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2 2023-03-23 08:55:23,441 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:23,441 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d2a2668ed83ec24163021b10f82d1b3e in 922ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:23,441 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:23,441 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:23,441 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:23,442 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:23,442 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,442 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-23 08:55:23,442 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-23 08:55:23,443 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,443 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:23,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:23,444 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,444 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:23,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:23,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:23,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:23,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:23,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-23 08:55:23,445 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c92b605[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:23,445 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-23 08:55:23,445 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,446 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,447 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-23 08:55:23,447 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-23 08:55:23,447 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:23,448 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-23 08:55:23,448 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,449 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-23 08:55:23,449 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:23,449 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-23 08:55:23,449 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:23,449 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:23,449 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-23 08:55:23,449 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:23,448 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:23,450 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-23 08:55:23,452 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,452 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,452 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:23,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:23,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:23,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:23,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:23,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,454 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:23,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:23,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:23,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:23,455 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:23,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:23,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:23,459 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:23,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:23,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:23,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-23 08:55:23,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:23,459 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-23 08:55:23,459 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,460 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,460 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:23,460 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,460 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:23,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:23,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:23,460 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:23,560 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:23,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:23,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:23,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:23,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:23,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:23,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:23,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-23 08:55:23,567 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-23 08:55:23,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:23,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-23 08:55:23,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-23 08:55:23,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,567 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:23,568 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,568 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:23,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:23,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:23,569 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:23,569 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:23,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:23,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:23,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:23,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:23,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-23 08:55:23,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-23 08:55:23,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-23 08:55:23,570 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-23 08:55:23,571 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:23,571 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-23 08:55:23,571 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:23,571 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:23,571 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. started... 2023-03-23 08:55:23,571 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. started... 2023-03-23 08:55:23,571 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8316b0845ad8dc2f32d05f84414a28c1 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:23,571 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d69f3c8983e2fde2ae01956e233272af 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:23,576 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:23,576 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:23,576 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:23,576 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. started... 2023-03-23 08:55:23,576 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d2a2668ed83ec24163021b10f82d1b3e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:23,998 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032311df6360a0db4800ba7bc6cef9f95d87 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032311df6360a0db4800ba7bc6cef9f95d87 2023-03-23 08:55:24,002 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032370103316582b42bd8295a9281b7d66ab to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032370103316582b42bd8295a9281b7d66ab 2023-03-23 08:55:24,002 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303237fb5c39ffb6a4d54984d9567f2f02788 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303237fb5c39ffb6a4d54984d9567f2f02788 2023-03-23 08:55:24,018 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/f2ce30f45b0c41b0a1d501d48db71001 2023-03-23 08:55:24,406 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/e6199993bf6d4b778e852ab64d2fc61b 2023-03-23 08:55:24,426 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/230c75ac9c0d46238fa9067ac3de958c 2023-03-23 08:55:24,441 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323cc48fa26a16c495baa04c33e69c8609d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230323cc48fa26a16c495baa04c33e69c8609d 2023-03-23 08:55:24,453 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323eaf6d8a31b6e4b518dc2f65dca75bf46 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230323eaf6d8a31b6e4b518dc2f65dca75bf46 2023-03-23 08:55:24,820 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303233058093b435a4c5d871316c2b96ef308 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303233058093b435a4c5d871316c2b96ef308 2023-03-23 08:55:24,825 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/df7e323e39864cf38c66a330a16f3a7f 2023-03-23 08:55:24,829 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/e6199993bf6d4b778e852ab64d2fc61b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b 2023-03-23 08:55:24,834 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:24,834 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/df7e323e39864cf38c66a330a16f3a7f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f 2023-03-23 08:55:24,839 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:24,840 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d69f3c8983e2fde2ae01956e233272af in 1269ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:24,840 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:24,840 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:24,840 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:24,847 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/7058de4f428d47eaa454d6519351cfdf 2023-03-23 08:55:24,852 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/f2ce30f45b0c41b0a1d501d48db71001 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001 2023-03-23 08:55:24,856 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:24,857 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/7058de4f428d47eaa454d6519351cfdf as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf 2023-03-23 08:55:24,857 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/9b71525cab694d24bcec04aab317a4f2 2023-03-23 08:55:24,864 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/230c75ac9c0d46238fa9067ac3de958c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c 2023-03-23 08:55:24,865 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:24,866 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d2a2668ed83ec24163021b10f82d1b3e in 1290ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:24,866 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:24,866 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:24,867 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:24,868 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c, entries=4, sequenceid=9, filesize=5.4 K 2023-03-23 08:55:24,868 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/9b71525cab694d24bcec04aab317a4f2 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2 2023-03-23 08:55:24,871 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:24,872 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8316b0845ad8dc2f32d05f84414a28c1 in 1301ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:24,872 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:24,872 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:24,872 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:24,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:24,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-23 08:55:24,873 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-23 08:55:24,874 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,875 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,875 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,875 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:24,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:24,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-23 08:55:24,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7537c4b0[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:24,877 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-23 08:55:24,877 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,880 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-23 08:55:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-23 08:55:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:24,880 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,880 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:24,885 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-23 08:55:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:24,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:24,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-23 08:55:24,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:24,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:24,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:24,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-23 08:55:24,887 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-23 08:55:24,887 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:24,887 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:24,887 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-23 08:55:24,888 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:24,889 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,889 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:24,889 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,890 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:24,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:24,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:24,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:24,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,891 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,894 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,894 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:24,894 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:24,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-23 08:55:24,894 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:24,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:24,894 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:24,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-23 08:55:24,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:24,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:24,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:24,895 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:24,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:24,995 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:24,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:24,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:24,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:24,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:25,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:25,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:25,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-23 08:55:25,010 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-23 08:55:25,010 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:25,010 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-23 08:55:25,010 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-23 08:55:25,010 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,010 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:25,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,012 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:25,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:25,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:25,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:25,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:25,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:25,013 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-23 08:55:25,013 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-23 08:55:25,013 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-23 08:55:25,013 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-23 08:55:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. started... 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:25,014 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8316b0845ad8dc2f32d05f84414a28c1 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. started... 2023-03-23 08:55:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:25,014 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d69f3c8983e2fde2ae01956e233272af 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:25,014 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:25,014 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. started... 2023-03-23 08:55:25,015 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d2a2668ed83ec24163021b10f82d1b3e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:25,056 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323de49bcc8818a43b9a1279cd0e2b89982 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323de49bcc8818a43b9a1279cd0e2b89982 2023-03-23 08:55:25,059 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303238dfcba42b1c4488bb44995d943b83b8c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303238dfcba42b1c4488bb44995d943b83b8c 2023-03-23 08:55:25,430 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032387a1f48dc20f461593af95cdce6ee24f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032387a1f48dc20f461593af95cdce6ee24f 2023-03-23 08:55:25,446 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/d7c840653d7b42bd8bf7e2e837255dea 2023-03-23 08:55:25,460 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323f14e475908bf46efb603a750cb5e4baf to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230323f14e475908bf46efb603a750cb5e4baf 2023-03-23 08:55:25,478 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/74b1007c8b324d71b06e415da0f44956 2023-03-23 08:55:25,483 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/d4dc39e9b97241229073f22e5ca932f5 2023-03-23 08:55:25,494 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e017cfd4d2f2486dbb245dc2e1716158 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230323e017cfd4d2f2486dbb245dc2e1716158 2023-03-23 08:55:25,500 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303239e2dd2baf46944dc98b93376259c5a29 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303239e2dd2baf46944dc98b93376259c5a29 2023-03-23 08:55:25,500 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/fa5286101617463083b482a2320143d9 2023-03-23 08:55:25,505 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/74b1007c8b324d71b06e415da0f44956 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956 2023-03-23 08:55:25,509 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:25,510 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/fa5286101617463083b482a2320143d9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9 2023-03-23 08:55:25,510 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/9f4f6c5f7c5548adbcdb396837e62d8a 2023-03-23 08:55:25,515 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:25,516 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d69f3c8983e2fde2ae01956e233272af in 502ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:25,516 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/d4dc39e9b97241229073f22e5ca932f5 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5 2023-03-23 08:55:25,516 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:25,516 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:25,516 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:25,520 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:25,521 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/9f4f6c5f7c5548adbcdb396837e62d8a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a 2023-03-23 08:55:25,526 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:25,526 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8316b0845ad8dc2f32d05f84414a28c1 in 512ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:25,526 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:25,526 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:25,526 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:25,865 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/0dcd3100588d4066827cfc27caf412eb 2023-03-23 08:55:25,870 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/d7c840653d7b42bd8bf7e2e837255dea as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea 2023-03-23 08:55:25,873 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea, entries=4, sequenceid=13, filesize=5.4 K 2023-03-23 08:55:25,874 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/0dcd3100588d4066827cfc27caf412eb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb 2023-03-23 08:55:25,878 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:25,879 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d2a2668ed83ec24163021b10f82d1b3e in 865ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:25,879 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:25,879 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:25,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:25,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:25,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-23 08:55:25,879 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-23 08:55:25,881 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,881 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:25,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:25,882 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,882 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:25,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:25,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:25,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:25,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-23 08:55:25,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c69755a[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:25,883 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-23 08:55:25,883 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,884 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,884 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-23 08:55:25,884 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-23 08:55:25,885 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:25,885 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,885 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:25,888 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-23 08:55:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:25,888 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-23 08:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:25,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:25,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:25,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-23 08:55:25,894 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-23 08:55:25,894 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:25,894 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:25,894 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-23 08:55:25,894 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:25,896 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,896 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,896 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:25,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,897 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:25,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:25,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:25,902 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:25,902 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:25,902 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:25,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-23 08:55:25,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:25,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:25,903 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:25,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:25,903 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:25,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:26,003 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:26,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:26,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:26,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:26,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:26,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:26,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:26,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-23 08:55:26,012 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-23 08:55:26,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:26,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-23 08:55:26,012 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-23 08:55:26,013 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:26,013 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:26,014 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:26,014 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:26,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:26,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:26,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:26,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:26,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-23 08:55:26,016 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-23 08:55:26,016 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-23 08:55:26,016 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-23 08:55:26,016 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:26,016 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-23 08:55:26,017 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:26,017 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. started... 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:26,020 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:26,020 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. started... 2023-03-23 08:55:26,020 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8316b0845ad8dc2f32d05f84414a28c1 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:26,020 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. started... 2023-03-23 08:55:26,020 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d2a2668ed83ec24163021b10f82d1b3e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:26,020 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d69f3c8983e2fde2ae01956e233272af 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:26,058 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323771a04765c2946bb9064b5913a30dc74 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230323771a04765c2946bb9064b5913a30dc74 2023-03-23 08:55:26,065 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/0fcba42b79a9494fa24a5e66dbdfe195 2023-03-23 08:55:26,463 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323c87d16ae62bc42dd84eaa2087b04516b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230323c87d16ae62bc42dd84eaa2087b04516b 2023-03-23 08:55:26,465 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303239667a3ceedaf4c0fa4ac7a0943bff7f8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303239667a3ceedaf4c0fa4ac7a0943bff7f8 2023-03-23 08:55:26,476 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/85015d7b8b7544a38b4789a10844df3d 2023-03-23 08:55:26,479 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323c8da502d2eda429881b8c58ac16f7829 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230323c8da502d2eda429881b8c58ac16f7829 2023-03-23 08:55:26,500 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323d3914321937e43168527022d99556b17 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230323d3914321937e43168527022d99556b17 2023-03-23 08:55:26,876 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/53188668d6e846b3b1e2969651516307 2023-03-23 08:55:26,891 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/24a40fab1a7a496cbc7308e77d4ea8c0 2023-03-23 08:55:26,896 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/0fcba42b79a9494fa24a5e66dbdfe195 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195 2023-03-23 08:55:26,899 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:26,900 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/24a40fab1a7a496cbc7308e77d4ea8c0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0 2023-03-23 08:55:26,903 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:26,904 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d69f3c8983e2fde2ae01956e233272af in 884ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:26,904 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:26,904 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:26,904 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:26,905 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/1de5ba7ba0d34d589aaddaac9001842c 2023-03-23 08:55:26,909 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/85015d7b8b7544a38b4789a10844df3d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d 2023-03-23 08:55:26,912 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:26,912 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/1de5ba7ba0d34d589aaddaac9001842c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c 2023-03-23 08:55:26,915 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:26,916 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d2a2668ed83ec24163021b10f82d1b3e in 896ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:26,916 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:26,916 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:26,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:27,298 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303233a8e179ea2cd49caa9f31d5d3283d88e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303233a8e179ea2cd49caa9f31d5d3283d88e 2023-03-23 08:55:27,636 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:27,703 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/1e943d1e2ca54f7ba9f1fde2b80a074c 2023-03-23 08:55:27,709 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/53188668d6e846b3b1e2969651516307 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307 2023-03-23 08:55:27,716 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307, entries=4, sequenceid=17, filesize=5.4 K 2023-03-23 08:55:27,717 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/1e943d1e2ca54f7ba9f1fde2b80a074c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c 2023-03-23 08:55:27,725 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:27,728 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8316b0845ad8dc2f32d05f84414a28c1 in 1708ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:27,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:27,728 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:27,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:27,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:27,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-23 08:55:27,728 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-23 08:55:27,737 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,737 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,738 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:27,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:27,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-23 08:55:27,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59f3488e[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:27,739 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-23 08:55:27,740 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,741 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,741 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-23 08:55:27,741 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-23 08:55:27,741 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:27,742 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,742 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:27,748 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:27,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-23 08:55:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:27,749 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-23 08:55:27,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:27,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-23 08:55:27,751 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-23 08:55:27,751 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:27,751 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:27,751 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-23 08:55:27,751 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:27,753 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,753 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,753 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:27,753 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,755 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:27,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:27,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:27,757 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:27,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-23 08:55:27,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:27,757 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,758 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,758 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:27,758 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,758 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:27,758 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:27,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:27,858 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:27,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:27,867 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:27,875 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:27,883 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:27,884 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:27,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:27,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:27,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-23 08:55:27,917 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-23 08:55:27,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:27,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-23 08:55:27,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-23 08:55:27,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:27,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,919 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:27,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:27,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:27,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:27,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:27,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:27,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-23 08:55:27,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-23 08:55:27,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-23 08:55:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-23 08:55:27,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-23 08:55:27,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:27,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:27,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. started... 2023-03-23 08:55:27,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. started... 2023-03-23 08:55:27,921 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing d69f3c8983e2fde2ae01956e233272af 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:27,921 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8316b0845ad8dc2f32d05f84414a28c1 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:27,921 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:27,922 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:27,922 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:27,922 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. started... 2023-03-23 08:55:27,922 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d2a2668ed83ec24163021b10f82d1b3e 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:28,749 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/57702e72bd5f464a8dfcf2ccb3f4f50e 2023-03-23 08:55:28,750 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/fa58eb57aab142a19ba1ccbbc8f785eb 2023-03-23 08:55:28,753 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57702e72bd5f464a8dfcf2ccb3f4f50e 2023-03-23 08:55:28,754 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/4891ce268e75405bbca725523e91be66 2023-03-23 08:55:28,754 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa58eb57aab142a19ba1ccbbc8f785eb 2023-03-23 08:55:28,757 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4891ce268e75405bbca725523e91be66 2023-03-23 08:55:29,173 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/46edf5f88b8043f88fc3cb42b8c3449c 2023-03-23 08:55:29,176 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46edf5f88b8043f88fc3cb42b8c3449c 2023-03-23 08:55:29,177 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/57702e72bd5f464a8dfcf2ccb3f4f50e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e 2023-03-23 08:55:29,181 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57702e72bd5f464a8dfcf2ccb3f4f50e 2023-03-23 08:55:29,181 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e, entries=3, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,181 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/68f2dfd7d71b4f77a0e143eb8c199ac4 2023-03-23 08:55:29,182 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/46edf5f88b8043f88fc3cb42b8c3449c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c 2023-03-23 08:55:29,184 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/de1e0087716b418199feb92851e0997a 2023-03-23 08:55:29,186 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68f2dfd7d71b4f77a0e143eb8c199ac4 2023-03-23 08:55:29,187 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/fa58eb57aab142a19ba1ccbbc8f785eb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb 2023-03-23 08:55:29,187 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46edf5f88b8043f88fc3cb42b8c3449c 2023-03-23 08:55:29,188 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,188 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d2a2668ed83ec24163021b10f82d1b3e in 1266ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:29,188 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:29,189 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:29,189 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:29,189 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de1e0087716b418199feb92851e0997a 2023-03-23 08:55:29,189 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/4891ce268e75405bbca725523e91be66 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66 2023-03-23 08:55:29,193 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa58eb57aab142a19ba1ccbbc8f785eb 2023-03-23 08:55:29,193 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb, entries=3, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,193 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/68f2dfd7d71b4f77a0e143eb8c199ac4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4 2023-03-23 08:55:29,194 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4891ce268e75405bbca725523e91be66 2023-03-23 08:55:29,195 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66, entries=3, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,195 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/de1e0087716b418199feb92851e0997a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a 2023-03-23 08:55:29,200 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68f2dfd7d71b4f77a0e143eb8c199ac4 2023-03-23 08:55:29,200 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,201 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8316b0845ad8dc2f32d05f84414a28c1 in 1280ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:29,201 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:29,201 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:29,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:29,204 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de1e0087716b418199feb92851e0997a 2023-03-23 08:55:29,204 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:29,205 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d69f3c8983e2fde2ae01956e233272af in 1284ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:29,205 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:29,205 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:29,205 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:29,205 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:29,205 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,205 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-23 08:55:29,205 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-23 08:55:29,214 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,214 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:29,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,215 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:29,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-23 08:55:29,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4048ef2b[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:29,216 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-23 08:55:29,216 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,218 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,218 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-23 08:55:29,218 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-23 08:55:29,218 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:29,218 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,218 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:29,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-23 08:55:29,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:29,222 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,222 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-23 08:55:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:29,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:29,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:29,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:29,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:29,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:29,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-23 08:55:29,226 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-23 08:55:29,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:29,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:29,226 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-23 08:55:29,226 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:29,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:29,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:29,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:29,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:29,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:29,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:29,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:29,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,239 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:29,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-23 08:55:29,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,241 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:29,241 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:29,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:29,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:29,241 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:29,242 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:29,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-23 08:55:29,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-23 08:55:29,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:29,242 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:29,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-23 08:55:29,342 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:29,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:29,344 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:29,344 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 8316b0845ad8dc2f32d05f84414a28c1/family1 is initiating major compaction (all files) 2023-03-23 08:55:29,344 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:29,344 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:29,345 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 8316b0845ad8dc2f32d05f84414a28c1/family1 in ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:29,345 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp, totalSize=26.6 K 2023-03-23 08:55:29,346 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 53b18717761e4d6fb5b51d697b903489, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722469 2023-03-23 08:55:29,346 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 230c75ac9c0d46238fa9067ac3de958c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:29,347 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting d4dc39e9b97241229073f22e5ca932f5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724998 2023-03-23 08:55:29,347 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 53188668d6e846b3b1e2969651516307, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:29,347 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting fa58eb57aab142a19ba1ccbbc8f785eb, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:29,357 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 8316b0845ad8dc2f32d05f84414a28c1#family1#compaction#150 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:29,767 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230323810b019e97694208bcdd4958889f6adc_del 2023-03-23 08:55:29,767 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323810b019e97694208bcdd4958889f6adc_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323810b019e97694208bcdd4958889f6adc_del 2023-03-23 08:55:30,579 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b07386d343648fc8625aa438f136bac 2023-03-23 08:55:30,580 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family1/7b07386d343648fc8625aa438f136bac as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/7b07386d343648fc8625aa438f136bac 2023-03-23 08:55:30,583 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7b07386d343648fc8625aa438f136bac 2023-03-23 08:55:30,583 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8316b0845ad8dc2f32d05f84414a28c1/family1 of 8316b0845ad8dc2f32d05f84414a28c1 into 7b07386d343648fc8625aa438f136bac(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:30,584 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:30,584 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:30,584 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 8316b0845ad8dc2f32d05f84414a28c1/family2 is initiating major compaction (all files) 2023-03-23 08:55:30,584 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:30,584 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:30,585 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 8316b0845ad8dc2f32d05f84414a28c1/family2 in ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:55:30,585 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp, totalSize=25.3 K 2023-03-23 08:55:30,585 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0113c8833f9349d895da7fb2387d6d18, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722469 2023-03-23 08:55:30,585 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 9b71525cab694d24bcec04aab317a4f2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:30,586 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 9f4f6c5f7c5548adbcdb396837e62d8a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724998 2023-03-23 08:55:30,586 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1e943d1e2ca54f7ba9f1fde2b80a074c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:30,586 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 68f2dfd7d71b4f77a0e143eb8c199ac4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:30,593 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 8316b0845ad8dc2f32d05f84414a28c1#family2#compaction#151 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:30,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb] to archive 2023-03-23 08:55:30,639 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:30,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53b18717761e4d6fb5b51d697b903489 2023-03-23 08:55:30,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/230c75ac9c0d46238fa9067ac3de958c 2023-03-23 08:55:30,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/d4dc39e9b97241229073f22e5ca932f5 2023-03-23 08:55:30,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/53188668d6e846b3b1e2969651516307 2023-03-23 08:55:30,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/fa58eb57aab142a19ba1ccbbc8f785eb 2023-03-23 08:55:31,001 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230323ecef7dd26a544a97b12883f9ea1d197f_del 2023-03-23 08:55:31,001 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323ecef7dd26a544a97b12883f9ea1d197f_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230323ecef7dd26a544a97b12883f9ea1d197f_del 2023-03-23 08:55:31,813 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ff727b36be7417a8fee15fdc2d53f9c 2023-03-23 08:55:31,813 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/.tmp/family2/5ff727b36be7417a8fee15fdc2d53f9c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/5ff727b36be7417a8fee15fdc2d53f9c 2023-03-23 08:55:31,817 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5ff727b36be7417a8fee15fdc2d53f9c 2023-03-23 08:55:31,817 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8316b0845ad8dc2f32d05f84414a28c1/family2 of 8316b0845ad8dc2f32d05f84414a28c1 into 5ff727b36be7417a8fee15fdc2d53f9c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:31,818 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:31,818 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:31,818 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): d69f3c8983e2fde2ae01956e233272af/family1 is initiating major compaction (all files) 2023-03-23 08:55:31,818 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:31,818 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:31,819 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of d69f3c8983e2fde2ae01956e233272af/family1 in ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:31,819 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp, totalSize=26.6 K 2023-03-23 08:55:31,819 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 30e89a86fae14b2ea22b0c846ee522b6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722468 2023-03-23 08:55:31,819 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting e6199993bf6d4b778e852ab64d2fc61b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:31,820 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 74b1007c8b324d71b06e415da0f44956, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724997 2023-03-23 08:55:31,820 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0fcba42b79a9494fa24a5e66dbdfe195, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:31,820 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 4891ce268e75405bbca725523e91be66, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:31,827 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): d69f3c8983e2fde2ae01956e233272af#family1#compaction#152 average throughput is 1.65 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:31,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4] to archive 2023-03-23 08:55:31,840 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:31,841 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/0113c8833f9349d895da7fb2387d6d18 2023-03-23 08:55:31,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9b71525cab694d24bcec04aab317a4f2 2023-03-23 08:55:31,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/9f4f6c5f7c5548adbcdb396837e62d8a 2023-03-23 08:55:31,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/1e943d1e2ca54f7ba9f1fde2b80a074c 2023-03-23 08:55:31,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family2/68f2dfd7d71b4f77a0e143eb8c199ac4 2023-03-23 08:55:32,234 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303236411765918e5493cafb8c39f3466a779_del 2023-03-23 08:55:32,234 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303236411765918e5493cafb8c39f3466a779_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303236411765918e5493cafb8c39f3466a779_del 2023-03-23 08:55:33,047 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7305fa62de90405aa7d7199284a883f1 2023-03-23 08:55:33,048 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family1/7305fa62de90405aa7d7199284a883f1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/7305fa62de90405aa7d7199284a883f1 2023-03-23 08:55:33,051 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7305fa62de90405aa7d7199284a883f1 2023-03-23 08:55:33,052 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d69f3c8983e2fde2ae01956e233272af/family1 of d69f3c8983e2fde2ae01956e233272af into 7305fa62de90405aa7d7199284a883f1(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:33,052 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:33,052 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:33,052 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): d69f3c8983e2fde2ae01956e233272af/family2 is initiating major compaction (all files) 2023-03-23 08:55:33,053 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:33,053 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:33,053 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of d69f3c8983e2fde2ae01956e233272af/family2 in ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:55:33,054 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp, totalSize=25.3 K 2023-03-23 08:55:33,054 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting ef6570714d3a4d418c21d82cba1926cb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722468 2023-03-23 08:55:33,054 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting df7e323e39864cf38c66a330a16f3a7f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:33,055 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting fa5286101617463083b482a2320143d9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724997 2023-03-23 08:55:33,055 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 24a40fab1a7a496cbc7308e77d4ea8c0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:33,055 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting de1e0087716b418199feb92851e0997a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:33,064 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): d69f3c8983e2fde2ae01956e233272af#family2#compaction#153 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:33,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66] to archive 2023-03-23 08:55:33,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:33,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/30e89a86fae14b2ea22b0c846ee522b6 2023-03-23 08:55:33,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/e6199993bf6d4b778e852ab64d2fc61b 2023-03-23 08:55:33,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/74b1007c8b324d71b06e415da0f44956 2023-03-23 08:55:33,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/0fcba42b79a9494fa24a5e66dbdfe195 2023-03-23 08:55:33,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/4891ce268e75405bbca725523e91be66 2023-03-23 08:55:33,473 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230323e722760c3bb34192ba3960e8b221e6ae_del 2023-03-23 08:55:33,473 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323e722760c3bb34192ba3960e8b221e6ae_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230323e722760c3bb34192ba3960e8b221e6ae_del 2023-03-23 08:55:34,285 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da2eed17f05c4a31acdf8c5e1102e567 2023-03-23 08:55:34,286 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/.tmp/family2/da2eed17f05c4a31acdf8c5e1102e567 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/da2eed17f05c4a31acdf8c5e1102e567 2023-03-23 08:55:34,289 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da2eed17f05c4a31acdf8c5e1102e567 2023-03-23 08:55:34,290 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d69f3c8983e2fde2ae01956e233272af/family2 of d69f3c8983e2fde2ae01956e233272af into da2eed17f05c4a31acdf8c5e1102e567(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:34,290 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:34,290 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:34,290 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): d2a2668ed83ec24163021b10f82d1b3e/family1 is initiating major compaction (all files) 2023-03-23 08:55:34,290 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:34,290 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:34,292 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of d2a2668ed83ec24163021b10f82d1b3e/family1 in ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:34,292 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp, totalSize=26.6 K 2023-03-23 08:55:34,292 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0cf7a9ba087e4a81ac2fd8718fc75768, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722470 2023-03-23 08:55:34,292 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting f2ce30f45b0c41b0a1d501d48db71001, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:34,293 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting d7c840653d7b42bd8bf7e2e837255dea, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724998 2023-03-23 08:55:34,293 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 85015d7b8b7544a38b4789a10844df3d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:34,293 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 57702e72bd5f464a8dfcf2ccb3f4f50e, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:34,302 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): d2a2668ed83ec24163021b10f82d1b3e#family1#compaction#154 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:34,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a] to archive 2023-03-23 08:55:34,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:34,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/ef6570714d3a4d418c21d82cba1926cb 2023-03-23 08:55:34,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/df7e323e39864cf38c66a330a16f3a7f 2023-03-23 08:55:34,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/fa5286101617463083b482a2320143d9 2023-03-23 08:55:34,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/24a40fab1a7a496cbc7308e77d4ea8c0 2023-03-23 08:55:34,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family2/de1e0087716b418199feb92851e0997a 2023-03-23 08:55:34,709 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323930ccc2c8e474a5ab9b15ce6f1d38cc0_del 2023-03-23 08:55:34,709 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323930ccc2c8e474a5ab9b15ce6f1d38cc0_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230323930ccc2c8e474a5ab9b15ce6f1d38cc0_del 2023-03-23 08:55:35,521 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a07c7f356de04ae9b3a20e369f47151e 2023-03-23 08:55:35,522 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family1/a07c7f356de04ae9b3a20e369f47151e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/a07c7f356de04ae9b3a20e369f47151e 2023-03-23 08:55:35,525 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a07c7f356de04ae9b3a20e369f47151e 2023-03-23 08:55:35,526 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2a2668ed83ec24163021b10f82d1b3e/family1 of d2a2668ed83ec24163021b10f82d1b3e into a07c7f356de04ae9b3a20e369f47151e(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:35,526 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:35,527 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:35,527 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): d2a2668ed83ec24163021b10f82d1b3e/family2 is initiating major compaction (all files) 2023-03-23 08:55:35,527 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:35,527 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:35,527 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of d2a2668ed83ec24163021b10f82d1b3e/family2 in ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:55:35,527 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp, totalSize=25.3 K 2023-03-23 08:55:35,527 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 4300e334fe734707a584895fe4736fd2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561722470 2023-03-23 08:55:35,527 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 7058de4f428d47eaa454d6519351cfdf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561723562 2023-03-23 08:55:35,528 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0dcd3100588d4066827cfc27caf412eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561724998 2023-03-23 08:55:35,528 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1de5ba7ba0d34d589aaddaac9001842c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561726005 2023-03-23 08:55:35,528 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 46edf5f88b8043f88fc3cb42b8c3449c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:35,535 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): d2a2668ed83ec24163021b10f82d1b3e#family2#compaction#155 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:35,538 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e] to archive 2023-03-23 08:55:35,539 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:35,540 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/0cf7a9ba087e4a81ac2fd8718fc75768 2023-03-23 08:55:35,541 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/f2ce30f45b0c41b0a1d501d48db71001 2023-03-23 08:55:35,542 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/d7c840653d7b42bd8bf7e2e837255dea 2023-03-23 08:55:35,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/85015d7b8b7544a38b4789a10844df3d 2023-03-23 08:55:35,543 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/57702e72bd5f464a8dfcf2ccb3f4f50e 2023-03-23 08:55:35,943 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323c6801894eab04d458a6f8ee630a5f412_del 2023-03-23 08:55:35,943 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323c6801894eab04d458a6f8ee630a5f412_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230323c6801894eab04d458a6f8ee630a5f412_del 2023-03-23 08:55:36,756 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 568a01ec17124f228cdde545abc7615c 2023-03-23 08:55:36,756 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/.tmp/family2/568a01ec17124f228cdde545abc7615c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/568a01ec17124f228cdde545abc7615c 2023-03-23 08:55:36,759 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 568a01ec17124f228cdde545abc7615c 2023-03-23 08:55:36,760 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d2a2668ed83ec24163021b10f82d1b3e/family2 of d2a2668ed83ec24163021b10f82d1b3e into 568a01ec17124f228cdde545abc7615c(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:36,760 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:36,763 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:36,766 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:36,768 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:36,769 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:36,774 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:55:36,787 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-23 08:55:36,790 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303236411765918e5493cafb8c39f3466a779_del 2023-03-23 08:55:36,793 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230323810b019e97694208bcdd4958889f6adc_del 2023-03-23 08:55:36,795 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323930ccc2c8e474a5ab9b15ce6f1d38cc0_del 2023-03-23 08:55:36,795 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:36,795 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(139): Connect 0x45b9ebc9 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:36,800 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7650a122, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:36,801 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:36,801 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:36,801 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:36,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c] to archive 2023-03-23 08:55:36,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:36,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/4300e334fe734707a584895fe4736fd2 2023-03-23 08:55:36,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/7058de4f428d47eaa454d6519351cfdf 2023-03-23 08:55:36,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/0dcd3100588d4066827cfc27caf412eb 2023-03-23 08:55:36,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/1de5ba7ba0d34d589aaddaac9001842c 2023-03-23 08:55:36,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family2/46edf5f88b8043f88fc3cb42b8c3449c 2023-03-23 08:55:37,243 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323e9619b49cf8c4ea6ad1f494cdb87fb68 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323e9619b49cf8c4ea6ad1f494cdb87fb68 2023-03-23 08:55:37,244 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,257 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:37,260 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45766, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:37,267 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:55:37,270 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46970, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:55:37,284 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230323/family1/efaf1d6f18bd4754acd33b18979c3129 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-23 08:55:37,285 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230323/family1/efaf1d6f18bd4754acd33b18979c3129}] 2023-03-23 08:55:37,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230323/family1/efaf1d6f18bd4754acd33b18979c3129 for inclusion in 8316b0845ad8dc2f32d05f84414a28c1/family1 2023-03-23 08:55:37,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-23 08:55:37,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-23 08:55:37,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:55:37,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230323/family1/efaf1d6f18bd4754acd33b18979c3129 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__stog57hk03ihajj0a3vt0qkpiuovm700r5c7cqnoev4tmhj0ojo6sumfnmr8tr6s/family1/efaf1d6f18bd4754acd33b18979c3129 2023-03-23 08:55:37,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__stog57hk03ihajj0a3vt0qkpiuovm700r5c7cqnoev4tmhj0ojo6sumfnmr8tr6s/family1/efaf1d6f18bd4754acd33b18979c3129 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/b7b89217ba454c07b889e729413be79e_SeqId_28_ 2023-03-23 08:55:37,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__stog57hk03ihajj0a3vt0qkpiuovm700r5c7cqnoev4tmhj0ojo6sumfnmr8tr6s/family1/efaf1d6f18bd4754acd33b18979c3129 into 8316b0845ad8dc2f32d05f84414a28c1/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/b7b89217ba454c07b889e729413be79e_SeqId_28_ - updating store file list. 2023-03-23 08:55:37,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/b7b89217ba454c07b889e729413be79e_SeqId_28_ into 8316b0845ad8dc2f32d05f84414a28c1/family1 2023-03-23 08:55:37,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__stog57hk03ihajj0a3vt0qkpiuovm700r5c7cqnoev4tmhj0ojo6sumfnmr8tr6s/family1/efaf1d6f18bd4754acd33b18979c3129 into 8316b0845ad8dc2f32d05f84414a28c1/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/family1/b7b89217ba454c07b889e729413be79e_SeqId_28_) 2023-03-23 08:55:37,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__stog57hk03ihajj0a3vt0qkpiuovm700r5c7cqnoev4tmhj0ojo6sumfnmr8tr6s/family1/efaf1d6f18bd4754acd33b18979c3129 2023-03-23 08:55:37,301 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,302 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:37,303 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323ba594341367b46cc9b2de57c627e6cca to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230323ba594341367b46cc9b2de57c627e6cca 2023-03-23 08:55:37,303 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-23 08:55:37,303 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230323' 2023-03-23 08:55:37,303 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:37,638 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303234db5b5117a7d46d7860bd1cdabd20eb6 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202303234db5b5117a7d46d7860bd1cdabd20eb6 2023-03-23 08:55:37,638 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323513a2ca6dfd94aa78d6c28b84487ce1f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230323513a2ca6dfd94aa78d6c28b84487ce1f 2023-03-23 08:55:37,641 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,641 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,676 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230323/family1/62b67ad6f7204148974fec138ef8fc2c first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-23 08:55:37,676 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/family1/aca099180edc4e9e881b0f3d1b9b70f4 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-23 08:55:37,677 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230323/family1/62b67ad6f7204148974fec138ef8fc2c}] 2023-03-23 08:55:37,677 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/family1/aca099180edc4e9e881b0f3d1b9b70f4}] 2023-03-23 08:55:37,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/family1/aca099180edc4e9e881b0f3d1b9b70f4 for inclusion in d69f3c8983e2fde2ae01956e233272af/family1 2023-03-23 08:55:37,681 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230323/family1/62b67ad6f7204148974fec138ef8fc2c for inclusion in d2a2668ed83ec24163021b10f82d1b3e/family1 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last=1 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:55:37,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230323/family1/62b67ad6f7204148974fec138ef8fc2c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s25n026vtnafvuo5tjkduh2i9d3fi0mpjc80hrdcadoaoq078gk0p6icktnahlkv/family1/62b67ad6f7204148974fec138ef8fc2c 2023-03-23 08:55:37,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230323/family1/aca099180edc4e9e881b0f3d1b9b70f4 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s7jsbmcjshe01cqutbmjfjk234g1u8b3lcsrboieoqtb3is4sj7e61sr6g7lgsnb/family1/aca099180edc4e9e881b0f3d1b9b70f4 2023-03-23 08:55:37,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s7jsbmcjshe01cqutbmjfjk234g1u8b3lcsrboieoqtb3is4sj7e61sr6g7lgsnb/family1/aca099180edc4e9e881b0f3d1b9b70f4 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/1c4913b060634075a139a139f33038c6_SeqId_28_ 2023-03-23 08:55:37,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s25n026vtnafvuo5tjkduh2i9d3fi0mpjc80hrdcadoaoq078gk0p6icktnahlkv/family1/62b67ad6f7204148974fec138ef8fc2c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/55e7b051f0d94ae6ae06ab932f740d75_SeqId_28_ 2023-03-23 08:55:37,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s7jsbmcjshe01cqutbmjfjk234g1u8b3lcsrboieoqtb3is4sj7e61sr6g7lgsnb/family1/aca099180edc4e9e881b0f3d1b9b70f4 into d69f3c8983e2fde2ae01956e233272af/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/1c4913b060634075a139a139f33038c6_SeqId_28_ - updating store file list. 2023-03-23 08:55:37,690 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s25n026vtnafvuo5tjkduh2i9d3fi0mpjc80hrdcadoaoq078gk0p6icktnahlkv/family1/62b67ad6f7204148974fec138ef8fc2c into d2a2668ed83ec24163021b10f82d1b3e/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/55e7b051f0d94ae6ae06ab932f740d75_SeqId_28_ - updating store file list. 2023-03-23 08:55:37,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/1c4913b060634075a139a139f33038c6_SeqId_28_ into d69f3c8983e2fde2ae01956e233272af/family1 2023-03-23 08:55:37,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s7jsbmcjshe01cqutbmjfjk234g1u8b3lcsrboieoqtb3is4sj7e61sr6g7lgsnb/family1/aca099180edc4e9e881b0f3d1b9b70f4 into d69f3c8983e2fde2ae01956e233272af/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/family1/1c4913b060634075a139a139f33038c6_SeqId_28_) 2023-03-23 08:55:37,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s7jsbmcjshe01cqutbmjfjk234g1u8b3lcsrboieoqtb3is4sj7e61sr6g7lgsnb/family1/aca099180edc4e9e881b0f3d1b9b70f4 2023-03-23 08:55:37,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/55e7b051f0d94ae6ae06ab932f740d75_SeqId_28_ into d2a2668ed83ec24163021b10f82d1b3e/family1 2023-03-23 08:55:37,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s25n026vtnafvuo5tjkduh2i9d3fi0mpjc80hrdcadoaoq078gk0p6icktnahlkv/family1/62b67ad6f7204148974fec138ef8fc2c into d2a2668ed83ec24163021b10f82d1b3e/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/family1/55e7b051f0d94ae6ae06ab932f740d75_SeqId_28_) 2023-03-23 08:55:37,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__ns_testMinorCompaction__s25n026vtnafvuo5tjkduh2i9d3fi0mpjc80hrdcadoaoq078gk0p6icktnahlkv/family1/62b67ad6f7204148974fec138ef8fc2c 2023-03-23 08:55:37,695 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,696 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,696 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:37,696 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:37,697 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230323d80f3265100845ad88ab17fe055549be to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230323d80f3265100845ad88ab17fe055549be 2023-03-23 08:55:37,697 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303234606ec5377ba49e88a4aa044e36bdb94 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303234606ec5377ba49e88a4aa044e36bdb94 2023-03-23 08:55:37,698 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-23 08:55:37,698 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-23 08:55:37,698 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230323' 2023-03-23 08:55:37,698 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230323' 2023-03-23 08:55:37,698 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:37,698 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230323 for table='ns:testMinorCompaction' and column='family1' 2023-03-23 08:55:37,698 INFO [Listener at localhost/38371] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-23 08:55:37,701 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:37,703 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:37,706 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:37,707 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:37,726 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=251 (was 284), OpenFileDescriptor=641 (was 733), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 371), ProcessCount=169 (was 169), AvailableMemoryMB=5591 (was 5615) 2023-03-23 08:55:37,741 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=251, OpenFileDescriptor=641, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=169, AvailableMemoryMB=5591 2023-03-23 08:55:37,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-23 08:55:37,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:37,745 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:55:37,745 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-23 08:55:37,746 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:55:37,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-23 08:55:37,748 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:37,749 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b empty. 2023-03-23 08:55:37,749 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:37,749 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-23 08:55:37,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-23 08:55:38,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-23 08:55:38,159 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-23 08:55:38,160 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2956fdc6f28aecbd88e7bf44df609d8b, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:38,165 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:38,165 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 2956fdc6f28aecbd88e7bf44df609d8b, disabling compactions & flushes 2023-03-23 08:55:38,165 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,165 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,165 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. after waiting 0 ms 2023-03-23 08:55:38,165 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,166 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,166 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:38,167 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:55:38,168 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561738168"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561738168"}]},"ts":"1679561738168"} 2023-03-23 08:55:38,170 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-23 08:55:38,170 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:55:38,171 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561738170"}]},"ts":"1679561738170"} 2023-03-23 08:55:38,171 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-23 08:55:38,174 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2956fdc6f28aecbd88e7bf44df609d8b, ASSIGN}] 2023-03-23 08:55:38,176 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2956fdc6f28aecbd88e7bf44df609d8b, ASSIGN 2023-03-23 08:55:38,176 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2956fdc6f28aecbd88e7bf44df609d8b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:38,327 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2956fdc6f28aecbd88e7bf44df609d8b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:38,328 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561738327"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561738327"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561738327"}]},"ts":"1679561738327"} 2023-03-23 08:55:38,329 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 2956fdc6f28aecbd88e7bf44df609d8b, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:38,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-23 08:55:38,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2956fdc6f28aecbd88e7bf44df609d8b, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.', STARTKEY => '', ENDKEY => ''} 2023-03-23 08:55:38,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:38,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,486 INFO [StoreOpener-2956fdc6f28aecbd88e7bf44df609d8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,487 DEBUG [StoreOpener-2956fdc6f28aecbd88e7bf44df609d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:38,487 DEBUG [StoreOpener-2956fdc6f28aecbd88e7bf44df609d8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:38,488 INFO [StoreOpener-2956fdc6f28aecbd88e7bf44df609d8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2956fdc6f28aecbd88e7bf44df609d8b columnFamilyName family1 2023-03-23 08:55:38,489 INFO [StoreOpener-2956fdc6f28aecbd88e7bf44df609d8b-1] regionserver.HStore(310): Store=2956fdc6f28aecbd88e7bf44df609d8b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:38,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:55:38,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:38,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2956fdc6f28aecbd88e7bf44df609d8b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71403046, jitterRate=0.06398829817771912}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-23 08:55:38,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:38,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., pid=114, masterSystemTime=1679561738481 2023-03-23 08:55:38,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2956fdc6f28aecbd88e7bf44df609d8b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:38,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679561738498"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561738498"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561738498"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561738498"}]},"ts":"1679561738498"} 2023-03-23 08:55:38,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-23 08:55:38,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 2956fdc6f28aecbd88e7bf44df609d8b, server=jenkins-hbase4.apache.org,32923,1679561616941 in 171 msec 2023-03-23 08:55:38,502 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-23 08:55:38,503 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=2956fdc6f28aecbd88e7bf44df609d8b, ASSIGN in 327 msec 2023-03-23 08:55:38,503 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:55:38,503 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561738503"}]},"ts":"1679561738503"} 2023-03-23 08:55:38,504 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-23 08:55:38,506 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:55:38,507 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 763 msec 2023-03-23 08:55:38,623 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-23 08:55:38,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-23 08:55:38,851 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-23 08:55:38,885 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:38,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,895 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:38,895 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:38,895 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:38,895 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:38,896 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,896 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,899 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:38,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:38,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:38,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:38,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:38,899 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:38,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:38,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:38,901 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:38,901 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:38,901 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:38,901 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:38,901 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:38,901 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:38,901 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:38,902 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-23 08:55:39,374 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306f37ce10429c44b18cf85c64dba1bd78 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511306f37ce10429c44b18cf85c64dba1bd78 2023-03-23 08:55:39,683 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:39,778 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/8431ac5bcda34378b117159c9b2e4301 2023-03-23 08:55:39,783 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/8431ac5bcda34378b117159c9b2e4301 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301 2023-03-23 08:55:39,787 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-23 08:55:39,788 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 886ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:39,789 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:39,789 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:39,789 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:39,789 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:39,789 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,789 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:39,789 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:39,791 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,791 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:39,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:39,791 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,791 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:39,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:39,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:39,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:39,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:39,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b91b1d6[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:39,793 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:39,793 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,794 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,794 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:39,794 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:39,794 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:39,794 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,794 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:39,796 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:39,796 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,796 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:39,796 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:39,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:39,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:39,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:39,799 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:39,799 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:39,799 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:39,799 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,799 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:39,802 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,802 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:39,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,802 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:39,802 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,804 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:39,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,806 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:39,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:39,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:39,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:39,806 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:39,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:39,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:39,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,807 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,907 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:39,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:39,913 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:39,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:39,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,915 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,915 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,917 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:39,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:39,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:39,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,918 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:39,918 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:39,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:39,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:39,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:39,921 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:39,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:39,921 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:39,921 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-23 08:55:40,333 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a1e9a5a073bc4e9c8b327ec3962522bc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a1e9a5a073bc4e9c8b327ec3962522bc 2023-03-23 08:55:40,339 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/21e6273686724356af01011aea3f78e9 2023-03-23 08:55:40,342 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/21e6273686724356af01011aea3f78e9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9 2023-03-23 08:55:40,345 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9, entries=1, sequenceid=9, filesize=5.0 K 2023-03-23 08:55:40,346 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 425ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:40,347 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:40,347 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,348 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:40,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:40,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c15c65b[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:40,351 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:40,352 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,357 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,358 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:40,358 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:40,358 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:40,358 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,358 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:40,359 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,359 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:40,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:40,361 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:40,361 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:40,361 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:40,362 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,362 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:40,363 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,363 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,363 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,364 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,367 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,369 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,369 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,369 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,369 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,370 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:40,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:40,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:40,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:40,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:40,370 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:40,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:40,470 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:40,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:40,475 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:40,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:40,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,477 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:40,477 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:40,477 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:40,477 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:40,478 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,478 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,482 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:40,482 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:40,483 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,483 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:40,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:40,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:40,484 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:40,484 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:40,485 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:40,897 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201b215723730a04f93bb9d276ac7d5b1f8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201b215723730a04f93bb9d276ac7d5b1f8 2023-03-23 08:55:40,903 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/a7f99844eeba41fdbea22d1cc146f0a8 2023-03-23 08:55:40,909 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/a7f99844eeba41fdbea22d1cc146f0a8 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8 2023-03-23 08:55:40,913 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8, entries=1, sequenceid=13, filesize=4.9 K 2023-03-23 08:55:40,913 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 428ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:40,913 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:40,914 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:40,914 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:40,915 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,915 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,916 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:40,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fe1506f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:40,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:40,917 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,918 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:40,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:40,918 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:40,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,919 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:40,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:40,920 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:40,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,920 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:40,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:40,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:40,927 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:40,927 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:40,927 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:40,927 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,927 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:40,929 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,929 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:40,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:40,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,929 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:40,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:40,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:40,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:40,930 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:40,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,932 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:40,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:40,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:40,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:40,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:40,935 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:40,935 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:40,935 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:40,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:40,935 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:40,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:40,935 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:40,935 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:41,035 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:41,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:41,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:41,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:41,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,042 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:41,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:41,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:41,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,042 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,044 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:41,044 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:41,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:41,044 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,044 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:41,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:41,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,045 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:41,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:41,046 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:41,046 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:41,046 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:41,046 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:41,462 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a36613a7ca20471d95491eb1119c546e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205a36613a7ca20471d95491eb1119c546e 2023-03-23 08:55:41,467 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/05a0418a12b04eb1bb9e555b888a4db5 2023-03-23 08:55:41,473 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/05a0418a12b04eb1bb9e555b888a4db5 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5 2023-03-23 08:55:41,479 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5, entries=1, sequenceid=17, filesize=4.9 K 2023-03-23 08:55:41,480 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 434ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:41,480 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:41,480 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:41,480 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:41,480 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:41,480 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,480 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:41,480 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:41,482 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,482 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,482 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,482 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:41,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:41,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8d7c70f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:41,485 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:41,492 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,496 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:41,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:41,496 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:41,496 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,496 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:41,497 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:41,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:41,498 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:41,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:41,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:41,501 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:41,501 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:41,501 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:41,501 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,501 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:41,503 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,503 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:41,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:41,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:41,504 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,505 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,507 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:41,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:41,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:41,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:41,509 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:41,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,509 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,509 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,509 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,509 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:41,509 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:41,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:41,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:41,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:41,609 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:41,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:41,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:41,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:41,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,617 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:41,617 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,617 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:41,617 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:41,618 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,618 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,620 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,620 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:41,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,620 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,620 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:41,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:41,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:41,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,621 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:41,621 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,621 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:41,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:41,622 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:41,622 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:41,622 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:41,622 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:41,636 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228817792728b944610809d889b484b57f7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228817792728b944610809d889b484b57f7 2023-03-23 08:55:41,654 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/226f860b55e74e7ca268de53c27e5a75 2023-03-23 08:55:41,658 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/226f860b55e74e7ca268de53c27e5a75 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75 2023-03-23 08:55:41,667 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75, entries=1, sequenceid=21, filesize=4.9 K 2023-03-23 08:55:41,668 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 46ms, sequenceid=21, compaction requested=true 2023-03-23 08:55:41,668 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:41,668 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:41,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:41,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:41,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:41,668 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:41,670 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,670 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,671 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,671 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:41,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:41,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ecec8c2[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:41,673 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:41,673 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,674 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,674 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:41,674 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:41,674 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:41,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,674 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:41,676 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:41,676 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:41,676 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:41,676 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:41,681 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:41,681 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:41,681 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:41,681 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:41,681 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,681 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:41,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,684 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:41,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:41,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:41,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,685 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:41,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:41,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:41,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,686 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:41,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:41,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,692 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:41,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:41,692 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:41,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:41,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:41,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:41,792 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:41,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:41,797 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:41,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:41,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,800 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:41,800 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,800 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:41,800 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:41,800 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,800 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,802 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,802 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:41,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:41,803 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:41,803 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,803 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,804 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:41,804 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:41,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:41,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:41,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:41,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:41,804 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:41,804 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:41,804 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:41,804 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:42,217 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a2a7b5dc971347ce9bf69f8800a5d034 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231a2a7b5dc971347ce9bf69f8800a5d034 2023-03-23 08:55:42,222 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/30155a2d91c24d399d3a50ef5213bb32 2023-03-23 08:55:42,226 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/30155a2d91c24d399d3a50ef5213bb32 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32 2023-03-23 08:55:42,229 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32, entries=1, sequenceid=25, filesize=4.9 K 2023-03-23 08:55:42,230 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 426ms, sequenceid=25, compaction requested=true 2023-03-23 08:55:42,230 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:42,230 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:42,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:42,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:42,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:42,231 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,233 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:42,233 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:42,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c8be05e[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:42,235 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:42,235 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,236 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:42,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,237 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:42,240 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:42,240 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,240 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,240 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:42,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:42,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:42,243 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:42,243 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:42,243 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:42,243 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,243 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:42,245 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:42,246 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,248 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,253 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:42,253 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:42,253 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:42,254 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:42,253 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:42,254 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:42,253 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:42,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:42,354 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:42,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:42,361 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:42,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:42,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,363 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:42,363 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:42,363 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:42,363 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:42,364 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,364 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,365 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,365 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:42,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:42,366 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,366 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:42,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:42,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,367 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:42,367 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:42,367 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:42,368 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:42,368 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:42,368 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:42,368 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:42,368 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:42,381 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f9cd277976634086a0b1289467579901 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101f9cd277976634086a0b1289467579901 2023-03-23 08:55:42,386 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/91578b2812304d65a3021b4e3f51cf7b 2023-03-23 08:55:42,390 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/91578b2812304d65a3021b4e3f51cf7b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b 2023-03-23 08:55:42,396 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b, entries=1, sequenceid=29, filesize=4.9 K 2023-03-23 08:55:42,397 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 29ms, sequenceid=29, compaction requested=true 2023-03-23 08:55:42,397 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:42,397 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:42,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:42,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:42,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:42,397 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:42,399 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,399 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,399 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,399 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:42,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:42,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@696cb788[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:42,401 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:42,401 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,404 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,404 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:42,404 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:42,404 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:42,404 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,404 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:42,406 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,406 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:42,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,406 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,406 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:42,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:42,409 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:42,409 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:42,409 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:42,409 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:42,409 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,409 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:42,411 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,411 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,411 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:42,412 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,413 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,416 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,416 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:42,416 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:42,416 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:42,422 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:42,422 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:42,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:42,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:42,516 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:42,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:42,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:42,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:42,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,523 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:42,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:42,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:42,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:42,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,523 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,526 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,526 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:42,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:42,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:42,526 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,526 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:42,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:42,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,527 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,527 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:42,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:42,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:42,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:42,528 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:42,528 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:42,528 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:42,528 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:42,528 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:42,941 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103791a7339bc9f4f5d9401f25ca4fead14 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103791a7339bc9f4f5d9401f25ca4fead14 2023-03-23 08:55:42,946 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/9f91be3e19ca4b83bb91fed45a838517 2023-03-23 08:55:42,950 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/9f91be3e19ca4b83bb91fed45a838517 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517 2023-03-23 08:55:42,954 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517, entries=1, sequenceid=33, filesize=4.9 K 2023-03-23 08:55:42,955 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 427ms, sequenceid=33, compaction requested=true 2023-03-23 08:55:42,955 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:42,955 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:42,955 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:42,955 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:42,955 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,955 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:42,955 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:42,957 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,957 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,958 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,958 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:42,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:42,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fb5ea15[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:42,959 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:42,959 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,960 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,960 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:42,960 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:42,960 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:42,962 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:42,962 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,962 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:42,962 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:42,962 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,962 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:42,964 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,964 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,964 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:42,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:42,965 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:42,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,966 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:42,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,972 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,972 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:42,972 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,972 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:42,972 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:42,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:42,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:42,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:42,972 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:42,973 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:42,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:42,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:43,073 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:43,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:43,077 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:43,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:43,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,082 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:43,082 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:43,082 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:43,082 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:43,083 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,083 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,084 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,084 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:43,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:43,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:43,084 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,084 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:43,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:43,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:43,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,086 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:43,086 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:43,086 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:43,086 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:43,086 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:43,086 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:43,086 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:43,087 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:43,502 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090717311f2c52f04ca793522992c1dd7b33 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015090717311f2c52f04ca793522992c1dd7b33 2023-03-23 08:55:43,512 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04 2023-03-23 08:55:43,518 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04 2023-03-23 08:55:43,523 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04, entries=1, sequenceid=37, filesize=4.9 K 2023-03-23 08:55:43,523 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 436ms, sequenceid=37, compaction requested=true 2023-03-23 08:55:43,523 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:43,524 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:43,524 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:43,527 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:43,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c5b1f9[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:43,530 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:43,531 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,531 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,532 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:43,532 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:43,532 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:43,532 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,532 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:43,533 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,533 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,533 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:43,533 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:43,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:43,536 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:43,536 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:43,536 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:43,536 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,536 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:43,537 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,537 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,537 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,538 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,539 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,541 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,541 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,542 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:43,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:43,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:43,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:43,542 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:43,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:43,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:43,643 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:43,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:43,647 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:43,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:43,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,649 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:43,649 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:43,649 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-23 08:55:43,649 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-23 08:55:43,649 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,649 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,650 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,650 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:43,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:43,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:43,650 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,650 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-23 08:55:43,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-23 08:55:43,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:43,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-23 08:55:43,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-23 08:55:43,652 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:43,652 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:43,652 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. started... 2023-03-23 08:55:43,652 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2956fdc6f28aecbd88e7bf44df609d8b 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-23 08:55:43,664 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511201e73325a414e4a5bb5cc96e585555590 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511201e73325a414e4a5bb5cc96e585555590 2023-03-23 08:55:43,669 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/da75634c98a8487bb3a015d4ad74d2f0 2023-03-23 08:55:43,676 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/da75634c98a8487bb3a015d4ad74d2f0 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0 2023-03-23 08:55:43,680 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0, entries=1, sequenceid=41, filesize=4.9 K 2023-03-23 08:55:43,681 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2956fdc6f28aecbd88e7bf44df609d8b in 29ms, sequenceid=41, compaction requested=true 2023-03-23 08:55:43,681 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:43,681 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:43,681 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-23 08:55:43,681 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-23 08:55:43,681 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,681 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-23 08:55:43,681 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-23 08:55:43,682 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,683 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,683 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,683 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:43,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-23 08:55:43,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14ff3c4f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:43,684 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-23 08:55:43,684 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,685 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,686 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-23 08:55:43,686 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-23 08:55:43,686 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:43,686 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,686 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:43,688 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,688 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-23 08:55:43,688 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:43,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,688 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-23 08:55:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:43,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-23 08:55:43,691 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-23 08:55:43,691 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:43,691 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:43,691 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,691 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:43,693 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,693 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,693 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:43,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:43,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:43,693 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:43,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:43,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:43,696 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:43,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:43,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-23 08:55:43,699 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:43,699 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-23 08:55:43,699 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:43,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:43,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:43,800 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:43,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:43,802 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-23 08:55:43,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:55:43,804 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:55:43,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:55:43,807 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562343807 2023-03-23 08:55:43,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-23 08:55:43,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-23 08:55:43,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:43,809 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x3d2ecbcd to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:55:43,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46508a35, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:55:43,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:44,031 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:44,249 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037301a7afe28f42ee8e6f24819214f595 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037301a7afe28f42ee8e6f24819214f595 2023-03-23 08:55:44,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:44,266 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:55:44,270 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38882, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:55:44,278 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:55:44,279 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37768, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:55:44,299 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 first=Optional[r5] last=Optional[r6] 2023-03-23 08:55:44,300 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9ae62f0f47fb474b866a0a12d2fdc2d7}] 2023-03-23 08:55:44,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 for inclusion in 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:44,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-23 08:55:44,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:55:44,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:44,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__84qc932akcgs56p9qok361igp4r8pt6esuap113gmk8d27pkg6cubs3mkvgcqbcl/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 2023-03-23 08:55:44,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__84qc932akcgs56p9qok361igp4r8pt6esuap113gmk8d27pkg6cubs3mkvgcqbcl/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_ 2023-03-23 08:55:44,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__84qc932akcgs56p9qok361igp4r8pt6esuap113gmk8d27pkg6cubs3mkvgcqbcl/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_ - updating store file list. 2023-03-23 08:55:44,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_ into 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:44,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__84qc932akcgs56p9qok361igp4r8pt6esuap113gmk8d27pkg6cubs3mkvgcqbcl/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_) 2023-03-23 08:55:44,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__84qc932akcgs56p9qok361igp4r8pt6esuap113gmk8d27pkg6cubs3mkvgcqbcl/family1/9ae62f0f47fb474b866a0a12d2fdc2d7 2023-03-23 08:55:44,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-23 08:55:44,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-23 08:55:44,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 2956fdc6f28aecbd88e7bf44df609d8b/family1 is initiating minor compaction 2023-03-23 08:55:44,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@73fb2dcb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:44,337 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2956fdc6f28aecbd88e7bf44df609d8b/family1 in testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:44,337 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp, totalSize=49.7 K 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 21e6273686724356af01011aea3f78e9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting a7f99844eeba41fdbea22d1cc146f0a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 05a0418a12b04eb1bb9e555b888a4db5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 226f860b55e74e7ca268de53c27e5a75, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 30155a2d91c24d399d3a50ef5213bb32, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 91578b2812304d65a3021b4e3f51cf7b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 9f91be3e19ca4b83bb91fed45a838517, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 5dcda82c193f4d0cb9ecb8d2a4ee2f04, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting da75634c98a8487bb3a015d4ad74d2f0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-23 08:55:44,337 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-23 08:55:44,339 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:44,340 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:44,343 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101f9cd277976634086a0b1289467579901 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101f9cd277976634086a0b1289467579901 2023-03-23 08:55:44,345 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103791a7339bc9f4f5d9401f25ca4fead14 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103791a7339bc9f4f5d9401f25ca4fead14 2023-03-23 08:55:44,345 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-23 08:55:44,345 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-23 08:55:44,345 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:44,346 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:44,384 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2956fdc6f28aecbd88e7bf44df609d8b#family1#compaction#166 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:44,807 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317589f104eba342d594e14ac15971d425 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317589f104eba342d594e14ac15971d425 2023-03-23 08:55:44,807 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/095a553339604b8ea1eb3b564c79fd73 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/095a553339604b8ea1eb3b564c79fd73 2023-03-23 08:55:44,808 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:44,814 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2956fdc6f28aecbd88e7bf44df609d8b/family1 of 2956fdc6f28aecbd88e7bf44df609d8b into 095a553339604b8ea1eb3b564c79fd73(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:44,814 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:44,814 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679561744336; duration=0sec 2023-03-23 08:55:44,814 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:44,841 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2dc99689d0844979a5e81d4a798f9782 first=Optional[r1] last=Optional[r4] 2023-03-23 08:55:44,843 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2dc99689d0844979a5e81d4a798f9782}] 2023-03-23 08:55:44,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_] to archive 2023-03-23 08:55:44,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:44,847 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2dc99689d0844979a5e81d4a798f9782 for inclusion in 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:44,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/21e6273686724356af01011aea3f78e9 2023-03-23 08:55:44,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-23 08:55:44,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:55:44,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:44,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/a7f99844eeba41fdbea22d1cc146f0a8 2023-03-23 08:55:44,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2dc99689d0844979a5e81d4a798f9782 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ka123vb6h4uv9oc3la8pd18r6hm63mgb6q6aea9ljpn9ovjhthd32kfhshbuvms2/family1/2dc99689d0844979a5e81d4a798f9782 2023-03-23 08:55:44,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/05a0418a12b04eb1bb9e555b888a4db5 2023-03-23 08:55:44,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/226f860b55e74e7ca268de53c27e5a75 2023-03-23 08:55:44,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/30155a2d91c24d399d3a50ef5213bb32 2023-03-23 08:55:44,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ka123vb6h4uv9oc3la8pd18r6hm63mgb6q6aea9ljpn9ovjhthd32kfhshbuvms2/family1/2dc99689d0844979a5e81d4a798f9782 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_ 2023-03-23 08:55:44,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/91578b2812304d65a3021b4e3f51cf7b 2023-03-23 08:55:44,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ka123vb6h4uv9oc3la8pd18r6hm63mgb6q6aea9ljpn9ovjhthd32kfhshbuvms2/family1/2dc99689d0844979a5e81d4a798f9782 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_ - updating store file list. 2023-03-23 08:55:44,863 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/9f91be3e19ca4b83bb91fed45a838517 2023-03-23 08:55:44,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5dcda82c193f4d0cb9ecb8d2a4ee2f04 2023-03-23 08:55:44,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_ into 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:44,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ka123vb6h4uv9oc3la8pd18r6hm63mgb6q6aea9ljpn9ovjhthd32kfhshbuvms2/family1/2dc99689d0844979a5e81d4a798f9782 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_) 2023-03-23 08:55:44,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/da75634c98a8487bb3a015d4ad74d2f0 2023-03-23 08:55:44,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ka123vb6h4uv9oc3la8pd18r6hm63mgb6q6aea9ljpn9ovjhthd32kfhshbuvms2/family1/2dc99689d0844979a5e81d4a798f9782 2023-03-23 08:55:44,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-23 08:55:44,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-23 08:55:44,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(1912): 2956fdc6f28aecbd88e7bf44df609d8b/family1 is initiating minor compaction (all files) 2023-03-23 08:55:44,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2789252c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:44,872 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2956fdc6f28aecbd88e7bf44df609d8b/family1 in testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:55:44,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/5516b0b5dc044afb86b7ecb89dbfa699_SeqId_44_ 2023-03-23 08:55:44,872 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/095a553339604b8ea1eb3b564c79fd73, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp, totalSize=178.1 K 2023-03-23 08:55:44,872 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 8431ac5bcda34378b117159c9b2e4301, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878841145 2023-03-23 08:55:44,873 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting 095a553339604b8ea1eb3b564c79fd73, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621241145 2023-03-23 08:55:44,873 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] compactions.Compactor(207): Compacting e56efe32bb8e431397abf18bcc55659c_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965241145 2023-03-23 08:55:44,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:44,876 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:44,878 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201b215723730a04f93bb9d276ac7d5b1f8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201b215723730a04f93bb9d276ac7d5b1f8 2023-03-23 08:55:44,880 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205a36613a7ca20471d95491eb1119c546e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205a36613a7ca20471d95491eb1119c546e 2023-03-23 08:55:44,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228817792728b944610809d889b484b57f7 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228817792728b944610809d889b484b57f7 2023-03-23 08:55:44,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231a2a7b5dc971347ce9bf69f8800a5d034 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231a2a7b5dc971347ce9bf69f8800a5d034 2023-03-23 08:55:44,885 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-23 08:55:44,885 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-23 08:55:44,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:44,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:44,907 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2956fdc6f28aecbd88e7bf44df609d8b#family1#compaction#167 average throughput is 6.48 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:44,973 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511305a7c36086eaf47eb8f6e4ca77bafd57c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511305a7c36086eaf47eb8f6e4ca77bafd57c 2023-03-23 08:55:44,974 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:45,002 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4d92f6289c014ee0bff433c0b3d1e852 first=Optional[r00] last=Optional[r8] 2023-03-23 08:55:45,003 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4d92f6289c014ee0bff433c0b3d1e852}] 2023-03-23 08:55:45,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4d92f6289c014ee0bff433c0b3d1e852 for inclusion in 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:45,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-23 08:55:45,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last= 2023-03-23 08:55:45,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:45,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/4d92f6289c014ee0bff433c0b3d1e852 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4jat0edbg92hepk0uktrsbhqoprvbdo19cga9860k9287hhdvv1hbko3sa65bjhh/family1/4d92f6289c014ee0bff433c0b3d1e852 2023-03-23 08:55:45,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4jat0edbg92hepk0uktrsbhqoprvbdo19cga9860k9287hhdvv1hbko3sa65bjhh/family1/4d92f6289c014ee0bff433c0b3d1e852 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/014ecec154824ffc9f81bf3eaae99636_SeqId_49_ 2023-03-23 08:55:45,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4jat0edbg92hepk0uktrsbhqoprvbdo19cga9860k9287hhdvv1hbko3sa65bjhh/family1/4d92f6289c014ee0bff433c0b3d1e852 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/014ecec154824ffc9f81bf3eaae99636_SeqId_49_ - updating store file list. 2023-03-23 08:55:45,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/014ecec154824ffc9f81bf3eaae99636_SeqId_49_ into 2956fdc6f28aecbd88e7bf44df609d8b/family1 2023-03-23 08:55:45,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4jat0edbg92hepk0uktrsbhqoprvbdo19cga9860k9287hhdvv1hbko3sa65bjhh/family1/4d92f6289c014ee0bff433c0b3d1e852 into 2956fdc6f28aecbd88e7bf44df609d8b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/014ecec154824ffc9f81bf3eaae99636_SeqId_49_) 2023-03-23 08:55:45,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4jat0edbg92hepk0uktrsbhqoprvbdo19cga9860k9287hhdvv1hbko3sa65bjhh/family1/4d92f6289c014ee0bff433c0b3d1e852 2023-03-23 08:55:45,018 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:45,018 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:45,019 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511201e73325a414e4a5bb5cc96e585555590 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511201e73325a414e4a5bb5cc96e585555590 2023-03-23 08:55:45,020 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a1e9a5a073bc4e9c8b327ec3962522bc to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128a1e9a5a073bc4e9c8b327ec3962522bc 2023-03-23 08:55:45,021 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511306f37ce10429c44b18cf85c64dba1bd78 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511306f37ce10429c44b18cf85c64dba1bd78 2023-03-23 08:55:45,021 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-23 08:55:45,021 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-23 08:55:45,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:45,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:45,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-23 08:55:45,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-23 08:55:45,022 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:55:45,024 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-23 08:55:45,025 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.2220 sec 2023-03-23 08:55:45,164 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:45,165 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3d2ecbcd to 127.0.0.1:50031 2023-03-23 08:55:45,165 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:45,165 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:45,165 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45b9ebc9 to 127.0.0.1:50031 2023-03-23 08:55:45,165 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:45,166 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:55:45,166 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1b29ae1b to 127.0.0.1:50031 2023-03-23 08:55:45,166 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:55:45,328 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=261 (was 251) - Thread LEAK? -, OpenFileDescriptor=659 (was 641) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=340 (was 339) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5521 (was 5591) 2023-03-23 08:55:45,341 INFO [Listener at localhost/38371] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=261, OpenFileDescriptor=659, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=169, AvailableMemoryMB=5520 2023-03-23 08:55:45,352 WARN [DataStreamer for file /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/715a4acd0f704e5e98a15875f04ec6f1] hdfs.DataStreamer(982): Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1257) at java.lang.Thread.join(Thread.java:1331) at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980) at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807) 2023-03-23 08:55:45,364 DEBUG [Listener at localhost/38371] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-23 08:55:45,365 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/.tmp/family1/715a4acd0f704e5e98a15875f04ec6f1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/715a4acd0f704e5e98a15875f04ec6f1 2023-03-23 08:55:45,370 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2956fdc6f28aecbd88e7bf44df609d8b/family1 of 2956fdc6f28aecbd88e7bf44df609d8b into 715a4acd0f704e5e98a15875f04ec6f1(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:45,370 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:55:45,370 INFO [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679561744871; duration=0sec 2023-03-23 08:55:45,370 DEBUG [RS:0;jenkins-hbase4:32923-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-23 08:55:45,425 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10O\xFCq8\x8Ez\x1Bj\xAF\x1E\xA2\xB3c\x9D\xE3\xD4"\x10\xA5^S\xA8r\x01\xFE\xB8\x9B\xD9\x0C\xF7\x1E\x02\x18\x16*\x10\xA7\xA0\x1FB\x9A\xCFV\xAE\xB3\xF9T\x9F\xE25\xA4\x912\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-23 08:55:45,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-23 08:55:45,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-23 08:55:45,428 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-23 08:55:45,428 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-23 08:55:45,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04 empty. 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b empty. 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c empty. 2023-03-23 08:55:45,431 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:45,432 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:45,432 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:45,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-23 08:55:45,442 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-23 08:55:45,444 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1f5319a61c93e4939d4243ead872bb04, NAME => 'testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10O\xFCq8\x8Ez\x1Bj\xAF\x1E\xA2\xB3c\x9D\xE3\xD4"\x10\xA5^S\xA8r\x01\xFE\xB8\x9B\xD9\x0C\xF7\x1E\x02\x18\x16*\x10\xA7\xA0\x1FB\x9A\xCFV\xAE\xB3\xF9T\x9F\xE25\xA4\x912\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:45,444 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6ce47bdacf42565a37db21d6312c1b4c, NAME => 'testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10O\xFCq8\x8Ez\x1Bj\xAF\x1E\xA2\xB3c\x9D\xE3\xD4"\x10\xA5^S\xA8r\x01\xFE\xB8\x9B\xD9\x0C\xF7\x1E\x02\x18\x16*\x10\xA7\xA0\x1FB\x9A\xCFV\xAE\xB3\xF9T\x9F\xE25\xA4\x912\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:45,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/095a553339604b8ea1eb3b564c79fd73, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_] to archive 2023-03-23 08:55:45,447 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => ab62bf5a7404efdd4302415b58f0194b, NAME => 'testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10O\xFCq8\x8Ez\x1Bj\xAF\x1E\xA2\xB3c\x9D\xE3\xD4"\x10\xA5^S\xA8r\x01\xFE\xB8\x9B\xD9\x0C\xF7\x1E\x02\x18\x16*\x10\xA7\xA0\x1FB\x9A\xCFV\xAE\xB3\xF9T\x9F\xE25\xA4\x912\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/.tmp 2023-03-23 08:55:45,447 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:45,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/8431ac5bcda34378b117159c9b2e4301 2023-03-23 08:55:45,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/095a553339604b8ea1eb3b564c79fd73 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/095a553339604b8ea1eb3b564c79fd73 2023-03-23 08:55:45,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_ to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/family1/e56efe32bb8e431397abf18bcc55659c_SeqId_47_ 2023-03-23 08:55:45,469 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:45,469 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing ab62bf5a7404efdd4302415b58f0194b, disabling compactions & flushes 2023-03-23 08:55:45,469 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:45,469 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:45,469 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. after waiting 0 ms 2023-03-23 08:55:45,470 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:45,470 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:45,470 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 6ce47bdacf42565a37db21d6312c1b4c, disabling compactions & flushes 2023-03-23 08:55:45,472 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. after waiting 0 ms 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:45,472 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:45,472 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:45,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-23 08:55:45,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 1f5319a61c93e4939d4243ead872bb04, disabling compactions & flushes 2023-03-23 08:55:45,863 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. after waiting 0 ms 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:45,863 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:45,863 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:45,866 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-23 08:55:45,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561745867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561745867"}]},"ts":"1679561745867"} 2023-03-23 08:55:45,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679561745867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561745867"}]},"ts":"1679561745867"} 2023-03-23 08:55:45,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561745867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679561745867"}]},"ts":"1679561745867"} 2023-03-23 08:55:45,875 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-23 08:55:45,876 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-23 08:55:45,876 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561745876"}]},"ts":"1679561745876"} 2023-03-23 08:55:45,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-23 08:55:45,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1f5319a61c93e4939d4243ead872bb04, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6ce47bdacf42565a37db21d6312c1b4c, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab62bf5a7404efdd4302415b58f0194b, ASSIGN}] 2023-03-23 08:55:45,884 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab62bf5a7404efdd4302415b58f0194b, ASSIGN 2023-03-23 08:55:45,884 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1f5319a61c93e4939d4243ead872bb04, ASSIGN 2023-03-23 08:55:45,885 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6ce47bdacf42565a37db21d6312c1b4c, ASSIGN 2023-03-23 08:55:45,885 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab62bf5a7404efdd4302415b58f0194b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:45,888 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1f5319a61c93e4939d4243ead872bb04, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:45,888 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6ce47bdacf42565a37db21d6312c1b4c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,32923,1679561616941; forceNewPlan=false, retain=false 2023-03-23 08:55:46,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-23 08:55:46,037 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=ab62bf5a7404efdd4302415b58f0194b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,037 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=6ce47bdacf42565a37db21d6312c1b4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,037 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561746037"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561746037"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561746037"}]},"ts":"1679561746037"} 2023-03-23 08:55:46,037 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679561746037"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561746037"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561746037"}]},"ts":"1679561746037"} 2023-03-23 08:55:46,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=1f5319a61c93e4939d4243ead872bb04, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,038 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561746037"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679561746037"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679561746037"}]},"ts":"1679561746037"} 2023-03-23 08:55:46,039 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure ab62bf5a7404efdd4302415b58f0194b, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:46,040 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 6ce47bdacf42565a37db21d6312c1b4c, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:46,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=117, state=RUNNABLE; OpenRegionProcedure 1f5319a61c93e4939d4243ead872bb04, server=jenkins-hbase4.apache.org,32923,1679561616941}] 2023-03-23 08:55:46,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1f5319a61c93e4939d4243ead872bb04, NAME => 'testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.', STARTKEY => '', ENDKEY => '1'} 2023-03-23 08:55:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,216 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,218 DEBUG [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1 2023-03-23 08:55:46,218 DEBUG [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1 2023-03-23 08:55:46,218 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f5319a61c93e4939d4243ead872bb04 columnFamilyName family1 2023-03-23 08:55:46,219 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] regionserver.HStore(310): Store=1f5319a61c93e4939d4243ead872bb04/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,219 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,220 DEBUG [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2 2023-03-23 08:55:46,221 DEBUG [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2 2023-03-23 08:55:46,221 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f5319a61c93e4939d4243ead872bb04 columnFamilyName family2 2023-03-23 08:55:46,222 INFO [StoreOpener-1f5319a61c93e4939d4243ead872bb04-1] regionserver.HStore(310): Store=1f5319a61c93e4939d4243ead872bb04/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:46,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:55:46,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:46,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1f5319a61c93e4939d4243ead872bb04; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75446696, jitterRate=0.12424337863922119}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:46,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:46,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04., pid=122, masterSystemTime=1679561746190 2023-03-23 08:55:46,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:46,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:46,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:46,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ce47bdacf42565a37db21d6312c1b4c, NAME => 'testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.', STARTKEY => '1', ENDKEY => '2'} 2023-03-23 08:55:46,230 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=1f5319a61c93e4939d4243ead872bb04, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,230 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561746229"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561746229"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561746229"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561746229"}]},"ts":"1679561746229"} 2023-03-23 08:55:46,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:46,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=117 2023-03-23 08:55:46,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=117, state=SUCCESS; OpenRegionProcedure 1f5319a61c93e4939d4243ead872bb04, server=jenkins-hbase4.apache.org,32923,1679561616941 in 191 msec 2023-03-23 08:55:46,233 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=1f5319a61c93e4939d4243ead872bb04, ASSIGN in 350 msec 2023-03-23 08:55:46,252 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,253 DEBUG [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1 2023-03-23 08:55:46,253 DEBUG [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1 2023-03-23 08:55:46,254 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ce47bdacf42565a37db21d6312c1b4c columnFamilyName family1 2023-03-23 08:55:46,255 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] regionserver.HStore(310): Store=6ce47bdacf42565a37db21d6312c1b4c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,255 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,256 DEBUG [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2 2023-03-23 08:55:46,256 DEBUG [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2 2023-03-23 08:55:46,257 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ce47bdacf42565a37db21d6312c1b4c columnFamilyName family2 2023-03-23 08:55:46,257 INFO [StoreOpener-6ce47bdacf42565a37db21d6312c1b4c-1] regionserver.HStore(310): Store=6ce47bdacf42565a37db21d6312c1b4c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:46,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:55:46,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:46,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6ce47bdacf42565a37db21d6312c1b4c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69650224, jitterRate=0.03786921501159668}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:46,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:46,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c., pid=121, masterSystemTime=1679561746190 2023-03-23 08:55:46,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:46,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:46,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:46,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ab62bf5a7404efdd4302415b58f0194b, NAME => 'testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.', STARTKEY => '2', ENDKEY => ''} 2023-03-23 08:55:46,266 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=6ce47bdacf42565a37db21d6312c1b4c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-23 08:55:46,266 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679561746266"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561746266"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561746266"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561746266"}]},"ts":"1679561746266"} 2023-03-23 08:55:46,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,270 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-23 08:55:46,270 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 6ce47bdacf42565a37db21d6312c1b4c, server=jenkins-hbase4.apache.org,32923,1679561616941 in 227 msec 2023-03-23 08:55:46,271 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6ce47bdacf42565a37db21d6312c1b4c, ASSIGN in 388 msec 2023-03-23 08:55:46,292 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,294 DEBUG [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1 2023-03-23 08:55:46,294 DEBUG [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1 2023-03-23 08:55:46,294 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab62bf5a7404efdd4302415b58f0194b columnFamilyName family1 2023-03-23 08:55:46,296 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] regionserver.HStore(310): Store=ab62bf5a7404efdd4302415b58f0194b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,297 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,298 DEBUG [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2 2023-03-23 08:55:46,298 DEBUG [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2 2023-03-23 08:55:46,298 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ab62bf5a7404efdd4302415b58f0194b columnFamilyName family2 2023-03-23 08:55:46,299 INFO [StoreOpener-ab62bf5a7404efdd4302415b58f0194b-1] regionserver.HStore(310): Store=ab62bf5a7404efdd4302415b58f0194b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-23 08:55:46,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-23 08:55:46,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:55:46,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-23 08:55:46,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ab62bf5a7404efdd4302415b58f0194b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66237849, jitterRate=-0.012979134917259216}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-23 08:55:46,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:46,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b., pid=120, masterSystemTime=1679561746190 2023-03-23 08:55:46,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:46,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:46,306 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=ab62bf5a7404efdd4302415b58f0194b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,307 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679561746306"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679561746306"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679561746306"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679561746306"}]},"ts":"1679561746306"} 2023-03-23 08:55:46,309 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-03-23 08:55:46,309 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure ab62bf5a7404efdd4302415b58f0194b, server=jenkins-hbase4.apache.org,32923,1679561616941 in 269 msec 2023-03-23 08:55:46,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-23 08:55:46,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ab62bf5a7404efdd4302415b58f0194b, ASSIGN in 427 msec 2023-03-23 08:55:46,311 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-23 08:55:46,311 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679561746311"}]},"ts":"1679561746311"} 2023-03-23 08:55:46,312 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-23 08:55:46,315 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-23 08:55:46,316 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 890 msec 2023-03-23 08:55:46,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-23 08:55:46,533 INFO [Listener at localhost/38371] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-23 08:55:46,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:46,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:46,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:46,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:46,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-23 08:55:46,545 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-23 08:55:46,545 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:46,545 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-23 08:55:46,545 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-23 08:55:46,545 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:46,545 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:46,549 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:46,549 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:46,549 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:46,549 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:46,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:46,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:46,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:46,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:46,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-23 08:55:46,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-23 08:55:46,552 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-23 08:55:46,552 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:46,552 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-23 08:55:46,552 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:46,552 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. started... 2023-03-23 08:55:46,554 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1f5319a61c93e4939d4243ead872bb04 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:46,554 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:46,554 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:46,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:46,554 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. started... 2023-03-23 08:55:46,554 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. started... 2023-03-23 08:55:46,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:46,555 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6ce47bdacf42565a37db21d6312c1b4c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:46,555 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ab62bf5a7404efdd4302415b58f0194b 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-23 08:55:46,620 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323ea61ac89b2b74ccaa1e4d460dfabed81 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323ea61ac89b2b74ccaa1e4d460dfabed81 2023-03-23 08:55:46,630 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303234c5ebefcf4974fe9afb36353b95f9400 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303234c5ebefcf4974fe9afb36353b95f9400 2023-03-23 08:55:46,643 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/333a59c4dd8b4a6f8e164b36f4e106ae 2023-03-23 08:55:47,018 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323bc757c9955bd47f19bdfa808f0d57426 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323bc757c9955bd47f19bdfa808f0d57426 2023-03-23 08:55:47,055 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/c673b4b3c84445aa953fffe9d7872418 2023-03-23 08:55:47,088 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323f73527f0615b48beb6957067c0b014d2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230323f73527f0615b48beb6957067c0b014d2 2023-03-23 08:55:47,102 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323111332633b6d447ba5490a3d0d7d674e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230323111332633b6d447ba5490a3d0d7d674e 2023-03-23 08:55:47,439 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/b15ddcce15a4488f8307b89f46a67dd9 2023-03-23 08:55:47,492 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/b64b7994126e4e35b7fb75071366d5fa 2023-03-23 08:55:47,503 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/333a59c4dd8b4a6f8e164b36f4e106ae as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae 2023-03-23 08:55:47,509 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/54acb850f9e34d6dbdd31dbb50d5e912 2023-03-23 08:55:47,513 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/c673b4b3c84445aa953fffe9d7872418 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418 2023-03-23 08:55:47,533 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae, entries=4, sequenceid=5, filesize=5.5 K 2023-03-23 08:55:47,534 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/b64b7994126e4e35b7fb75071366d5fa as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa 2023-03-23 08:55:47,540 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:47,541 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ab62bf5a7404efdd4302415b58f0194b in 986ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:47,541 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-23 08:55:47,542 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:47,542 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:47,542 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:47,547 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418, entries=4, sequenceid=5, filesize=5.5 K 2023-03-23 08:55:47,548 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/54acb850f9e34d6dbdd31dbb50d5e912 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912 2023-03-23 08:55:47,552 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:47,552 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 1f5319a61c93e4939d4243ead872bb04 in 998ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:47,553 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:47,554 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:47,887 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303232ab13da94fc742009b21cf39380bb72f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303232ab13da94fc742009b21cf39380bb72f 2023-03-23 08:55:47,892 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/04b7f48aeb574aee991b2d70c418e41a 2023-03-23 08:55:47,900 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/b15ddcce15a4488f8307b89f46a67dd9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9 2023-03-23 08:55:47,934 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9, entries=4, sequenceid=5, filesize=5.5 K 2023-03-23 08:55:47,934 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/04b7f48aeb574aee991b2d70c418e41a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a 2023-03-23 08:55:47,942 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a, entries=2, sequenceid=5, filesize=5.1 K 2023-03-23 08:55:47,942 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6ce47bdacf42565a37db21d6312c1b4c in 1387ms, sequenceid=5, compaction requested=false 2023-03-23 08:55:47,942 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:47,943 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:47,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:47,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:47,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-23 08:55:47,943 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-23 08:55:47,944 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,944 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:47,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:47,945 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:47,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-23 08:55:47,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@182319af[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:47,946 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-23 08:55:47,946 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,947 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,947 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-23 08:55:47,947 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-23 08:55:47,947 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,947 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:47,947 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:47,948 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,948 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-23 08:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:47,949 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:47,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:47,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:47,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-23 08:55:47,951 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-23 08:55:47,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:47,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:47,951 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-23 08:55:47,951 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:47,954 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,954 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:47,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,954 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:47,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:47,955 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,956 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,958 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,958 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:47,958 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:47,958 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:47,958 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:47,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-23 08:55:47,959 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:47,958 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:47,959 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:48,059 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:48,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:48,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:48,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:48,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:48,064 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:48,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:48,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-23 08:55:48,066 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-23 08:55:48,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:48,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-23 08:55:48,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-23 08:55:48,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:48,066 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:48,068 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:48,068 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:48,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:48,068 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:48,068 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:48,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:48,069 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:48,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:48,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-23 08:55:48,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-23 08:55:48,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-23 08:55:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-23 08:55:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-23 08:55:48,072 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:48,072 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. started... 2023-03-23 08:55:48,072 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:48,072 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1f5319a61c93e4939d4243ead872bb04 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:48,072 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:48,072 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:48,072 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:48,072 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. started... 2023-03-23 08:55:48,073 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. started... 2023-03-23 08:55:48,073 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ab62bf5a7404efdd4302415b58f0194b 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:48,073 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6ce47bdacf42565a37db21d6312c1b4c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-23 08:55:48,539 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323e9b4a1c5bc694b78914a345e8d652483 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323e9b4a1c5bc694b78914a345e8d652483 2023-03-23 08:55:48,540 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323a3e59710ee364178aed777b231237102 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a3e59710ee364178aed777b231237102 2023-03-23 08:55:48,549 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032339cb42d623e7468d8b41f580ad007b86 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032339cb42d623e7468d8b41f580ad007b86 2023-03-23 08:55:48,567 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/1a0f73a336224fb09323efbc0e2d7fd5 2023-03-23 08:55:48,973 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/3e753b53d39441d9aad67176e26e330a 2023-03-23 08:55:48,975 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/3765730e64f9432dae9a1104c3ab3678 2023-03-23 08:55:49,017 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303233754995a0584437d87be6747186beafa to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303233754995a0584437d87be6747186beafa 2023-03-23 08:55:49,046 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/83dd160d7dbc4ec99e6fb4375130011c 2023-03-23 08:55:49,055 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303238740c06e2585483488cc6ab265f578b2 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303238740c06e2585483488cc6ab265f578b2 2023-03-23 08:55:49,055 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323048b93042ac84104b641cbc9be7adf05 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230323048b93042ac84104b641cbc9be7adf05 2023-03-23 08:55:49,056 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/1a0f73a336224fb09323efbc0e2d7fd5 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5 2023-03-23 08:55:49,082 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/b10995288db54ed2940936b3b666061d 2023-03-23 08:55:49,086 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/3765730e64f9432dae9a1104c3ab3678 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678 2023-03-23 08:55:49,103 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5, entries=4, sequenceid=9, filesize=5.5 K 2023-03-23 08:55:49,104 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/83dd160d7dbc4ec99e6fb4375130011c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c 2023-03-23 08:55:49,110 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:49,111 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ab62bf5a7404efdd4302415b58f0194b in 1038ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:49,111 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:49,111 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:49,111 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:49,122 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678, entries=4, sequenceid=9, filesize=5.5 K 2023-03-23 08:55:49,123 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/b10995288db54ed2940936b3b666061d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d 2023-03-23 08:55:49,126 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:49,127 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6ce47bdacf42565a37db21d6312c1b4c in 1054ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:49,127 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:49,127 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:49,127 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:49,470 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/16400a58d7b740da8d78a7c1af19358c 2023-03-23 08:55:49,475 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/3e753b53d39441d9aad67176e26e330a as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a 2023-03-23 08:55:49,500 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a, entries=4, sequenceid=9, filesize=5.5 K 2023-03-23 08:55:49,501 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/16400a58d7b740da8d78a7c1af19358c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c 2023-03-23 08:55:49,505 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-23 08:55:49,506 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 1f5319a61c93e4939d4243ead872bb04 in 1433ms, sequenceid=9, compaction requested=false 2023-03-23 08:55:49,506 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:49,506 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:49,506 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:49,506 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:49,506 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,506 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-23 08:55:49,506 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-23 08:55:49,507 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,507 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:49,508 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,508 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:49,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-23 08:55:49,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6836bdfc[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:49,509 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-23 08:55:49,509 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,511 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,511 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-23 08:55:49,511 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-23 08:55:49,511 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,511 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:49,511 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:49,513 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-23 08:55:49,513 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:49,513 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-23 08:55:49,513 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:49,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:49,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-23 08:55:49,516 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-23 08:55:49,516 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:49,516 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:49,516 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-23 08:55:49,516 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:49,517 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,517 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:49,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:49,517 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:49,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:49,518 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:49,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:49,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:49,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,519 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:49,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:49,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:49,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:49,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:49,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:49,521 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:49,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:49,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:49,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:49,521 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,621 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:49,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:49,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:49,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:49,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:49,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:49,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:49,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-23 08:55:49,628 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-23 08:55:49,628 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:49,628 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-23 08:55:49,628 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-23 08:55:49,628 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,628 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:49,630 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,630 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:49,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:49,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:49,630 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:49,630 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:49,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:49,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:49,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:49,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:49,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-23 08:55:49,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-23 08:55:49,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-23 08:55:49,631 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-23 08:55:49,631 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:49,631 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-23 08:55:49,631 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:49,631 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:49,632 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. started... 2023-03-23 08:55:49,632 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. started... 2023-03-23 08:55:49,632 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1f5319a61c93e4939d4243ead872bb04 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:49,632 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:49,632 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:49,632 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ab62bf5a7404efdd4302415b58f0194b 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:49,632 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:49,632 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. started... 2023-03-23 08:55:49,632 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6ce47bdacf42565a37db21d6312c1b4c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-23 08:55:50,112 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303231954a49c8dee4faab8721961bc5680ed to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303231954a49c8dee4faab8721961bc5680ed 2023-03-23 08:55:50,127 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323a77ad661ffcf4ddb90a5ee5e9289d293 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a77ad661ffcf4ddb90a5ee5e9289d293 2023-03-23 08:55:50,129 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323a9bc1c869bdf4687b6b4d860a5d2c154 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a9bc1c869bdf4687b6b4d860a5d2c154 2023-03-23 08:55:50,142 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/b2356431849148d88c5da185e3f1fa1e 2023-03-23 08:55:50,157 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/cd685013fd01400b8f6b69f81924c2e8 2023-03-23 08:55:50,161 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/fbacc7c478e944baa0d993478d46e867 2023-03-23 08:55:50,223 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323ac9d13c9e9794b98986d913f4f702071 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230323ac9d13c9e9794b98986d913f4f702071 2023-03-23 08:55:50,235 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/ff867c2d4edc43f98ce7bb50279a1233 2023-03-23 08:55:50,237 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032328ab3de131354910aba952df341455c5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023032328ab3de131354910aba952df341455c5 2023-03-23 08:55:50,239 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/b2356431849148d88c5da185e3f1fa1e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e 2023-03-23 08:55:50,249 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/0ce59e05db184523951cd154b1323548 2023-03-23 08:55:50,253 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/cd685013fd01400b8f6b69f81924c2e8 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8 2023-03-23 08:55:50,267 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e, entries=4, sequenceid=13, filesize=5.5 K 2023-03-23 08:55:50,268 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/ff867c2d4edc43f98ce7bb50279a1233 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233 2023-03-23 08:55:50,271 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:50,272 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ab62bf5a7404efdd4302415b58f0194b in 640ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:50,272 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:50,272 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:50,285 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8, entries=4, sequenceid=13, filesize=5.5 K 2023-03-23 08:55:50,286 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/0ce59e05db184523951cd154b1323548 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548 2023-03-23 08:55:50,290 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:50,291 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 1f5319a61c93e4939d4243ead872bb04 in 659ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:50,291 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:50,291 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:50,291 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:50,631 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323189954ca09664209acc04e50cd83c443 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230323189954ca09664209acc04e50cd83c443 2023-03-23 08:55:51,036 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/293cbc9656dc4988a0afd9f1ef37d20e 2023-03-23 08:55:51,040 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/fbacc7c478e944baa0d993478d46e867 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867 2023-03-23 08:55:51,064 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867, entries=4, sequenceid=13, filesize=5.5 K 2023-03-23 08:55:51,065 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/293cbc9656dc4988a0afd9f1ef37d20e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e 2023-03-23 08:55:51,069 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e, entries=2, sequenceid=13, filesize=5.1 K 2023-03-23 08:55:51,070 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6ce47bdacf42565a37db21d6312c1b4c in 1438ms, sequenceid=13, compaction requested=true 2023-03-23 08:55:51,070 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:51,070 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:51,070 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:51,070 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:51,070 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,070 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-23 08:55:51,070 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-23 08:55:51,071 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,071 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:51,072 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,072 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:51,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-23 08:55:51,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62f6937a[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:51,073 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-23 08:55:51,073 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,074 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,074 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-23 08:55:51,074 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-23 08:55:51,074 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,075 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:51,074 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:51,077 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,077 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-23 08:55:51,077 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:51,077 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-23 08:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:51,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-23 08:55:51,079 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-23 08:55:51,079 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:51,079 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:51,079 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-23 08:55:51,079 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:51,081 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,081 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:51,081 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:51,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:51,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:51,083 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:51,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,085 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:51,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:51,087 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:51,087 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-23 08:55:51,087 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:51,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:51,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:51,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:51,087 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:51,188 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:51,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:51,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. with WAL disabled. Data may be lost in the event of a crash. 2023-03-23 08:55:51,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:51,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:51,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-23 08:55:51,194 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-23 08:55:51,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:51,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-23 08:55:51,194 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-23 08:55:51,195 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,195 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:51,200 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,200 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:51,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:51,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:51,200 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:51,200 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:51,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:51,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:51,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:51,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:51,201 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-23 08:55:51,201 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-23 08:55:51,201 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-23 08:55:51,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-23 08:55:51,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:51,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. started... 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:51,202 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. started... 2023-03-23 08:55:51,202 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1f5319a61c93e4939d4243ead872bb04 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:51,202 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. started... 2023-03-23 08:55:51,202 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6ce47bdacf42565a37db21d6312c1b4c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:51,202 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:51,202 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ab62bf5a7404efdd4302415b58f0194b 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-23 08:55:51,254 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032335a758f654d846da8e3ee5a665525a56 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032335a758f654d846da8e3ee5a665525a56 2023-03-23 08:55:51,281 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/a300b45e39614a91ad23cbfd94ea20ee 2023-03-23 08:55:51,294 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303233b6a49790aea4f828d4681a04c83c509 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303233b6a49790aea4f828d4681a04c83c509 2023-03-23 08:55:51,680 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323a58719334964443ebdb7dd16006e9476 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a58719334964443ebdb7dd16006e9476 2023-03-23 08:55:51,695 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-23 08:55:51,715 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/92855ee49fdc4816b777acd5110f7424 2023-03-23 08:55:51,732 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032319c33d02a45447a4aa8827d9ef21f9aa to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023032319c33d02a45447a4aa8827d9ef21f9aa 2023-03-23 08:55:51,755 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/c6cd166ee3674b8690e23892e90ecd44 2023-03-23 08:55:51,759 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/a300b45e39614a91ad23cbfd94ea20ee as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee 2023-03-23 08:55:51,794 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee, entries=4, sequenceid=17, filesize=5.5 K 2023-03-23 08:55:51,795 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/c6cd166ee3674b8690e23892e90ecd44 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44 2023-03-23 08:55:51,806 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:51,807 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 1f5319a61c93e4939d4243ead872bb04 in 605ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:51,807 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:51,807 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:51,807 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:52,100 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/dd013f14005141769e981d77bf265a8f 2023-03-23 08:55:52,149 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032311be8da5121d4000b922e13f2f25690f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023032311be8da5121d4000b922e13f2f25690f 2023-03-23 08:55:52,157 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/7c6f604c0f8c41cb9a37846abaf2ba1e 2023-03-23 08:55:52,164 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/dd013f14005141769e981d77bf265a8f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f 2023-03-23 08:55:52,194 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323a216a28ec8b040a5b371e79d8c41dafe to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230323a216a28ec8b040a5b371e79d8c41dafe 2023-03-23 08:55:52,195 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f, entries=4, sequenceid=17, filesize=5.5 K 2023-03-23 08:55:52,196 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/7c6f604c0f8c41cb9a37846abaf2ba1e as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e 2023-03-23 08:55:52,199 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/f1c097bcebff4876b3ac29f2e4e5bf8f 2023-03-23 08:55:52,200 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:52,201 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6ce47bdacf42565a37db21d6312c1b4c in 998ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:52,201 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:52,201 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:52,201 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:52,204 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/92855ee49fdc4816b777acd5110f7424 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424 2023-03-23 08:55:52,237 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424, entries=4, sequenceid=17, filesize=5.5 K 2023-03-23 08:55:52,239 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/f1c097bcebff4876b3ac29f2e4e5bf8f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f 2023-03-23 08:55:52,244 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-23 08:55:52,244 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ab62bf5a7404efdd4302415b58f0194b in 1042ms, sequenceid=17, compaction requested=true 2023-03-23 08:55:52,245 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:52,245 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:52,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:52,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:52,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-23 08:55:52,245 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-23 08:55:52,246 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,246 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:52,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:52,247 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,247 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:52,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:52,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-23 08:55:52,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f06c62f[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:52,250 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-23 08:55:52,250 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,251 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-23 08:55:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-23 08:55:52,251 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:52,251 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,251 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:52,253 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-23 08:55:52,253 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:52,253 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:52,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:52,253 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-23 08:55:52,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:52,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:52,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-23 08:55:52,258 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-23 08:55:52,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:52,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:52,258 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-23 08:55:52,258 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:52,262 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,262 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:52,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:52,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,263 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:52,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:52,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:52,264 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:52,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,266 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:52,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:52,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,270 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-23 08:55:52,270 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:52,270 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:52,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:52,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-23 08:55:52,270 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:52,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:52,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:52,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:52,371 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:52,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:52,386 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:52,401 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:52,411 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:55:52,412 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:55:52,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-23 08:55:52,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-23 08:55:52,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-23 08:55:52,506 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-23 08:55:52,506 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:52,506 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-23 08:55:52,506 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-23 08:55:52,507 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,507 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:52,508 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:52,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:52,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:52,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:52,508 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-23 08:55:52,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:52,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:52,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-23 08:55:52,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:52,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-23 08:55:52,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-23 08:55:52,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-23 08:55:52,514 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-23 08:55:52,514 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-23 08:55:52,514 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-23 08:55:52,514 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:52,514 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. started... 2023-03-23 08:55:52,514 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:52,514 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-23 08:55:52,514 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-23 08:55:52,514 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1f5319a61c93e4939d4243ead872bb04 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:52,515 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. started... 2023-03-23 08:55:52,514 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:52,515 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. started... 2023-03-23 08:55:52,515 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ab62bf5a7404efdd4302415b58f0194b 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:52,515 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6ce47bdacf42565a37db21d6312c1b4c 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-23 08:55:52,983 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/6ec57fd1c60f4cf6a9926512ac8d9298 2023-03-23 08:55:52,990 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/27b2d7edb74b4fda86c259fee1e3c52d 2023-03-23 08:55:53,012 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ec57fd1c60f4cf6a9926512ac8d9298 2023-03-23 08:55:53,021 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27b2d7edb74b4fda86c259fee1e3c52d 2023-03-23 08:55:53,396 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/56e5352dae6d4245b58f54bd285c37e9 2023-03-23 08:55:53,422 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 56e5352dae6d4245b58f54bd285c37e9 2023-03-23 08:55:53,433 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/055b61246e56491587c30c1d16540a95 2023-03-23 08:55:53,436 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 055b61246e56491587c30c1d16540a95 2023-03-23 08:55:53,437 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/6ec57fd1c60f4cf6a9926512ac8d9298 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298 2023-03-23 08:55:53,444 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/9ebc5ad2cab24bc29ac1cac667e658dd 2023-03-23 08:55:53,447 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ebc5ad2cab24bc29ac1cac667e658dd 2023-03-23 08:55:53,447 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/27b2d7edb74b4fda86c259fee1e3c52d as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d 2023-03-23 08:55:53,467 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ec57fd1c60f4cf6a9926512ac8d9298 2023-03-23 08:55:53,468 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298, entries=3, sequenceid=23, filesize=5.2 K 2023-03-23 08:55:53,468 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/055b61246e56491587c30c1d16540a95 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95 2023-03-23 08:55:53,471 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27b2d7edb74b4fda86c259fee1e3c52d 2023-03-23 08:55:53,471 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d, entries=3, sequenceid=23, filesize=5.2 K 2023-03-23 08:55:53,472 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/9ebc5ad2cab24bc29ac1cac667e658dd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd 2023-03-23 08:55:53,473 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 055b61246e56491587c30c1d16540a95 2023-03-23 08:55:53,473 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:53,475 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ab62bf5a7404efdd4302415b58f0194b in 960ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:53,475 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:53,475 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:53,475 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-23 08:55:53,477 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ebc5ad2cab24bc29ac1cac667e658dd 2023-03-23 08:55:53,477 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:53,478 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6ce47bdacf42565a37db21d6312c1b4c in 963ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:53,478 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:53,478 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:53,478 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-23 08:55:54,240 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/1f2ed4210c4b40899996ccd65f8a340b 2023-03-23 08:55:54,254 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f2ed4210c4b40899996ccd65f8a340b 2023-03-23 08:55:54,255 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/56e5352dae6d4245b58f54bd285c37e9 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9 2023-03-23 08:55:54,299 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 56e5352dae6d4245b58f54bd285c37e9 2023-03-23 08:55:54,299 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9, entries=3, sequenceid=23, filesize=5.2 K 2023-03-23 08:55:54,300 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/1f2ed4210c4b40899996ccd65f8a340b as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b 2023-03-23 08:55:54,327 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1f2ed4210c4b40899996ccd65f8a340b 2023-03-23 08:55:54,328 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b, entries=1, sequenceid=23, filesize=5.0 K 2023-03-23 08:55:54,329 INFO [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 1f5319a61c93e4939d4243ead872bb04 in 1815ms, sequenceid=23, compaction requested=true 2023-03-23 08:55:54,329 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:54,329 DEBUG [rs(jenkins-hbase4.apache.org,32923,1679561616941)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-23 08:55:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-23 08:55:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-23 08:55:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-23 08:55:54,331 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:54,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,331 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-23 08:55:54,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:54,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,32923,1679561616941' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-23 08:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fe85801[Count = 0] remaining members to acquire global barrier 2023-03-23 08:55:54,333 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-23 08:55:54,333 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,334 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-23 08:55:54,334 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,335 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-23 08:55:54,334 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-23 08:55:54,335 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,32923,1679561616941' in zk 2023-03-23 08:55:54,337 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-23 08:55:54,337 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:54,337 DEBUG [member: 'jenkins-hbase4.apache.org,32923,1679561616941' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-23 08:55:54,337 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,32923,1679561616941': 2023-03-23 08:55:54,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,32923,1679561616941' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-23 08:55:54,345 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-23 08:55:54,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-23 08:55:54,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-23 08:55:54,345 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-23 08:55:54,345 INFO [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-23 08:55:54,348 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,348 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:54,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,348 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-23 08:55:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-23 08:55:54,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:54,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-23 08:55:54,350 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-23 08:55:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,351 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-23 08:55:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-23 08:55:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,355 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-23 08:55:54,355 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:55:54,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-23 08:55:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-23 08:55:54,356 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-23 08:55:54,358 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-23 08:55:54,358 DEBUG [(jenkins-hbase4.apache.org,37881,1679561615669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-23 08:55:54,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-23 08:55:54,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-23 08:55:54,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-23 08:55:54,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-23 08:55:54,358 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-23 08:55:54,358 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-23 08:55:54,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-23 08:55:54,459 DEBUG [Listener at localhost/38371] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-23 08:55:54,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37881] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-23 08:55:54,460 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:54,460 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 1f5319a61c93e4939d4243ead872bb04/family1 is initiating major compaction (all files) 2023-03-23 08:55:54,460 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:54,460 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:54,461 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 1f5319a61c93e4939d4243ead872bb04/family1 in testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:54,461 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp, totalSize=27.1 K 2023-03-23 08:55:54,461 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting c673b4b3c84445aa953fffe9d7872418, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746539 2023-03-23 08:55:54,462 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 3e753b53d39441d9aad67176e26e330a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748061 2023-03-23 08:55:54,462 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting cd685013fd01400b8f6b69f81924c2e8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749623 2023-03-23 08:55:54,463 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting a300b45e39614a91ad23cbfd94ea20ee, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:54,463 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 56e5352dae6d4245b58f54bd285c37e9, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:54,474 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 1f5319a61c93e4939d4243ead872bb04#family1#compaction#198 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:54,514 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del 2023-03-23 08:55:54,514 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del 2023-03-23 08:55:54,567 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7454b8f15f446729d0379ab0d769dd7 2023-03-23 08:55:54,568 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family1/b7454b8f15f446729d0379ab0d769dd7 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/b7454b8f15f446729d0379ab0d769dd7 2023-03-23 08:55:54,591 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b7454b8f15f446729d0379ab0d769dd7 2023-03-23 08:55:54,592 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1f5319a61c93e4939d4243ead872bb04/family1 of 1f5319a61c93e4939d4243ead872bb04 into b7454b8f15f446729d0379ab0d769dd7(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:54,592 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:54,592 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:54,592 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 1f5319a61c93e4939d4243ead872bb04/family2 is initiating major compaction (all files) 2023-03-23 08:55:54,592 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:54,592 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:54,593 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 1f5319a61c93e4939d4243ead872bb04/family2 in testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:55:54,593 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp, totalSize=25.4 K 2023-03-23 08:55:54,593 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 54acb850f9e34d6dbdd31dbb50d5e912, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746539 2023-03-23 08:55:54,594 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 16400a58d7b740da8d78a7c1af19358c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748061 2023-03-23 08:55:54,594 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 0ce59e05db184523951cd154b1323548, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749623 2023-03-23 08:55:54,594 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting c6cd166ee3674b8690e23892e90ecd44, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:54,595 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1f2ed4210c4b40899996ccd65f8a340b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:54,603 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 1f5319a61c93e4939d4243ead872bb04#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:54,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9] to archive 2023-03-23 08:55:54,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:54,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/c673b4b3c84445aa953fffe9d7872418 2023-03-23 08:55:54,642 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/3e753b53d39441d9aad67176e26e330a 2023-03-23 08:55:54,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/cd685013fd01400b8f6b69f81924c2e8 2023-03-23 08:55:54,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/a300b45e39614a91ad23cbfd94ea20ee 2023-03-23 08:55:54,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/56e5352dae6d4245b58f54bd285c37e9 2023-03-23 08:55:55,018 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303231745f2689f3947a8b9582c0cedbafdf9_del 2023-03-23 08:55:55,019 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303231745f2689f3947a8b9582c0cedbafdf9_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303231745f2689f3947a8b9582c0cedbafdf9_del 2023-03-23 08:55:55,835 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f48e8dd2ad6e47bd8bc1bc897b5bf453 2023-03-23 08:55:55,835 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/.tmp/family2/f48e8dd2ad6e47bd8bc1bc897b5bf453 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/f48e8dd2ad6e47bd8bc1bc897b5bf453 2023-03-23 08:55:55,839 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f48e8dd2ad6e47bd8bc1bc897b5bf453 2023-03-23 08:55:55,840 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 1f5319a61c93e4939d4243ead872bb04/family2 of 1f5319a61c93e4939d4243ead872bb04 into f48e8dd2ad6e47bd8bc1bc897b5bf453(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:55,840 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:55:55,840 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:55,841 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): ab62bf5a7404efdd4302415b58f0194b/family1 is initiating major compaction (all files) 2023-03-23 08:55:55,841 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:55,841 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:55,842 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of ab62bf5a7404efdd4302415b58f0194b/family1 in testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:55,842 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp, totalSize=27.1 K 2023-03-23 08:55:55,843 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 333a59c4dd8b4a6f8e164b36f4e106ae, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746540 2023-03-23 08:55:55,843 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 1a0f73a336224fb09323efbc0e2d7fd5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748062 2023-03-23 08:55:55,843 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting b2356431849148d88c5da185e3f1fa1e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749624 2023-03-23 08:55:55,844 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 92855ee49fdc4816b777acd5110f7424, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:55,844 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 6ec57fd1c60f4cf6a9926512ac8d9298, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:55,853 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): ab62bf5a7404efdd4302415b58f0194b#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:55,925 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del 2023-03-23 08:55:55,925 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del 2023-03-23 08:55:55,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b] to archive 2023-03-23 08:55:55,939 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:55,941 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/54acb850f9e34d6dbdd31dbb50d5e912 2023-03-23 08:55:55,942 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/16400a58d7b740da8d78a7c1af19358c 2023-03-23 08:55:55,944 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/0ce59e05db184523951cd154b1323548 2023-03-23 08:55:55,945 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/c6cd166ee3674b8690e23892e90ecd44 2023-03-23 08:55:55,946 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family2/1f2ed4210c4b40899996ccd65f8a340b 2023-03-23 08:55:56,794 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c4f20b23d0942e6b0d1a12d4efb4920 2023-03-23 08:55:56,795 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family1/0c4f20b23d0942e6b0d1a12d4efb4920 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/0c4f20b23d0942e6b0d1a12d4efb4920 2023-03-23 08:55:56,828 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c4f20b23d0942e6b0d1a12d4efb4920 2023-03-23 08:55:56,829 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ab62bf5a7404efdd4302415b58f0194b/family1 of ab62bf5a7404efdd4302415b58f0194b into 0c4f20b23d0942e6b0d1a12d4efb4920(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:56,829 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:56,829 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:56,829 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): ab62bf5a7404efdd4302415b58f0194b/family2 is initiating major compaction (all files) 2023-03-23 08:55:56,829 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:56,829 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:56,830 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of ab62bf5a7404efdd4302415b58f0194b/family2 in testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:55:56,830 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp, totalSize=25.4 K 2023-03-23 08:55:56,832 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting b64b7994126e4e35b7fb75071366d5fa, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746540 2023-03-23 08:55:56,833 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 83dd160d7dbc4ec99e6fb4375130011c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748062 2023-03-23 08:55:56,833 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting ff867c2d4edc43f98ce7bb50279a1233, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749624 2023-03-23 08:55:56,833 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting f1c097bcebff4876b3ac29f2e4e5bf8f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:56,834 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 055b61246e56491587c30c1d16540a95, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:56,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298] to archive 2023-03-23 08:55:56,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:56,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/333a59c4dd8b4a6f8e164b36f4e106ae 2023-03-23 08:55:56,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/1a0f73a336224fb09323efbc0e2d7fd5 2023-03-23 08:55:56,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2356431849148d88c5da185e3f1fa1e 2023-03-23 08:55:56,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/92855ee49fdc4816b777acd5110f7424 2023-03-23 08:55:56,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/6ec57fd1c60f4cf6a9926512ac8d9298 2023-03-23 08:55:56,864 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): ab62bf5a7404efdd4302415b58f0194b#family2#compaction#201 average throughput is 0.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:56,888 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323e5720cd7b48f444799aebba308a874d3_del 2023-03-23 08:55:56,889 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230323e5720cd7b48f444799aebba308a874d3_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230323e5720cd7b48f444799aebba308a874d3_del 2023-03-23 08:55:57,737 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c36cfcf740234dd0a9c7b23c5e5a3575 2023-03-23 08:55:57,737 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/.tmp/family2/c36cfcf740234dd0a9c7b23c5e5a3575 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/c36cfcf740234dd0a9c7b23c5e5a3575 2023-03-23 08:55:57,740 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c36cfcf740234dd0a9c7b23c5e5a3575 2023-03-23 08:55:57,741 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ab62bf5a7404efdd4302415b58f0194b/family2 of ab62bf5a7404efdd4302415b58f0194b into c36cfcf740234dd0a9c7b23c5e5a3575(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-23 08:55:57,741 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:55:57,742 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:57,742 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 6ce47bdacf42565a37db21d6312c1b4c/family1 is initiating major compaction (all files) 2023-03-23 08:55:57,742 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:57,742 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:57,742 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 6ce47bdacf42565a37db21d6312c1b4c/family1 in testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:57,742 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp, totalSize=27.1 K 2023-03-23 08:55:57,742 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting b15ddcce15a4488f8307b89f46a67dd9, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746540 2023-03-23 08:55:57,743 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 3765730e64f9432dae9a1104c3ab3678, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748061 2023-03-23 08:55:57,743 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting fbacc7c478e944baa0d993478d46e867, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749624 2023-03-23 08:55:57,743 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting dd013f14005141769e981d77bf265a8f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:57,744 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 27b2d7edb74b4fda86c259fee1e3c52d, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:57,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95] to archive 2023-03-23 08:55:57,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:57,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/b64b7994126e4e35b7fb75071366d5fa 2023-03-23 08:55:57,749 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/83dd160d7dbc4ec99e6fb4375130011c 2023-03-23 08:55:57,750 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/ff867c2d4edc43f98ce7bb50279a1233 2023-03-23 08:55:57,751 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/f1c097bcebff4876b3ac29f2e4e5bf8f 2023-03-23 08:55:57,752 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 6ce47bdacf42565a37db21d6312c1b4c#family1#compaction#202 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:57,752 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family2/055b61246e56491587c30c1d16540a95 2023-03-23 08:55:58,189 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del 2023-03-23 08:55:58,189 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del 2023-03-23 08:55:59,045 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4aeefe9d8b1a4ef0aaafb87e201fbeda 2023-03-23 08:55:59,046 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family1/4aeefe9d8b1a4ef0aaafb87e201fbeda as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/4aeefe9d8b1a4ef0aaafb87e201fbeda 2023-03-23 08:55:59,067 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4aeefe9d8b1a4ef0aaafb87e201fbeda 2023-03-23 08:55:59,068 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6ce47bdacf42565a37db21d6312c1b4c/family1 of 6ce47bdacf42565a37db21d6312c1b4c into 4aeefe9d8b1a4ef0aaafb87e201fbeda(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:55:59,068 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:55:59,069 DEBUG [Listener at localhost/38371] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-23 08:55:59,069 DEBUG [Listener at localhost/38371] regionserver.HStore(1912): 6ce47bdacf42565a37db21d6312c1b4c/family2 is initiating major compaction (all files) 2023-03-23 08:55:59,069 INFO [Listener at localhost/38371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-23 08:55:59,069 INFO [Listener at localhost/38371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-23 08:55:59,069 INFO [Listener at localhost/38371] regionserver.HRegion(2259): Starting compaction of 6ce47bdacf42565a37db21d6312c1b4c/family2 in testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:55:59,069 INFO [Listener at localhost/38371] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd] into tmpdir=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp, totalSize=25.4 K 2023-03-23 08:55:59,069 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 04b7f48aeb574aee991b2d70c418e41a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679561746540 2023-03-23 08:55:59,070 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting b10995288db54ed2940936b3b666061d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679561748061 2023-03-23 08:55:59,070 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 293cbc9656dc4988a0afd9f1ef37d20e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679561749624 2023-03-23 08:55:59,070 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 7c6f604c0f8c41cb9a37846abaf2ba1e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679561751190 2023-03-23 08:55:59,071 DEBUG [Listener at localhost/38371] compactions.Compactor(207): Compacting 9ebc5ad2cab24bc29ac1cac667e658dd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-23 08:55:59,078 INFO [Listener at localhost/38371] throttle.PressureAwareThroughputController(145): 6ce47bdacf42565a37db21d6312c1b4c#family2#compaction#203 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-23 08:55:59,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d] to archive 2023-03-23 08:55:59,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:55:59,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/b15ddcce15a4488f8307b89f46a67dd9 2023-03-23 08:55:59,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/3765730e64f9432dae9a1104c3ab3678 2023-03-23 08:55:59,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/fbacc7c478e944baa0d993478d46e867 2023-03-23 08:55:59,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/dd013f14005141769e981d77bf265a8f 2023-03-23 08:55:59,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/27b2d7edb74b4fda86c259fee1e3c52d 2023-03-23 08:55:59,484 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303232640c30fee184f7aa2e90aee3a26f44f_del 2023-03-23 08:55:59,484 INFO [Listener at localhost/38371] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303232640c30fee184f7aa2e90aee3a26f44f_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303232640c30fee184f7aa2e90aee3a26f44f_del 2023-03-23 08:56:00,296 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 887dddcdd5394b74b741cb199651edd3 2023-03-23 08:56:00,297 DEBUG [Listener at localhost/38371] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/.tmp/family2/887dddcdd5394b74b741cb199651edd3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/887dddcdd5394b74b741cb199651edd3 2023-03-23 08:56:00,300 INFO [Listener at localhost/38371] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 887dddcdd5394b74b741cb199651edd3 2023-03-23 08:56:00,301 INFO [Listener at localhost/38371] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6ce47bdacf42565a37db21d6312c1b4c/family2 of 6ce47bdacf42565a37db21d6312c1b4c into 887dddcdd5394b74b741cb199651edd3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-23 08:56:00,301 DEBUG [Listener at localhost/38371] regionserver.HRegion(2289): Compaction status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:56:00,303 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:56:00,306 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:56:00,309 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:56:00,309 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:56:00,314 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37881] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10O\xFCq8\x8Ez\x1Bj\xAF\x1E\xA2\xB3c\x9D\xE3\xD4"\x10\xA5^S\xA8r\x01\xFE\xB8\x9B\xD9\x0C\xF7\x1E\x02\x18\x16*\x10\xA7\xA0\x1FB\x9A\xCFV\xAE\xB3\xF9T\x9F\xE25\xA4\x912\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-23 08:56:00,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-23 08:56:00,335 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-23 08:56:00,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:56:00,335 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679562360335 2023-03-23 08:56:00,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-23 08:56:00,339 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd] to archive 2023-03-23 08:56:00,340 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:00,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/04b7f48aeb574aee991b2d70c418e41a 2023-03-23 08:56:00,343 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/b10995288db54ed2940936b3b666061d 2023-03-23 08:56:00,344 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/293cbc9656dc4988a0afd9f1ef37d20e 2023-03-23 08:56:00,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/7c6f604c0f8c41cb9a37846abaf2ba1e 2023-03-23 08:56:00,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family2/9ebc5ad2cab24bc29ac1cac667e658dd 2023-03-23 08:56:00,631 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-23 08:56:00,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del 2023-03-23 08:56:00,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del 2023-03-23 08:56:00,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del 2023-03-23 08:56:00,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:00,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] zookeeper.ReadOnlyZKClient(139): Connect 0x2ea8dd42 to 127.0.0.1:50031 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-23 08:56:00,696 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6da86b10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-23 08:56:00,696 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:01,641 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230323fc40960932ba44a8a77d0900746c8c1b to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323fc40960932ba44a8a77d0900746c8c1b 2023-03-23 08:56:01,642 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:01,653 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-23 08:56:01,655 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58746, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-23 08:56:01,665 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-23 08:56:01,666 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41434, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-23 08:56:01,697 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230323/family1/e048604627d14e3795dad104af37306f first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-23 08:56:01,698 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230323/family1/e048604627d14e3795dad104af37306f}] 2023-03-23 08:56:01,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230323/family1/e048604627d14e3795dad104af37306f for inclusion in 6ce47bdacf42565a37db21d6312c1b4c/family1 2023-03-23 08:56:01,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-23 08:56:01,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-23 08:56:01,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:56:01,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230323/family1/e048604627d14e3795dad104af37306f to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__c8nderk1phs0el1ldm1vqfgu9jebf1ikk2t99k28267c7tg4855dfdrnlq3n2r5a/family1/e048604627d14e3795dad104af37306f 2023-03-23 08:56:01,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__c8nderk1phs0el1ldm1vqfgu9jebf1ikk2t99k28267c7tg4855dfdrnlq3n2r5a/family1/e048604627d14e3795dad104af37306f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/9086c2ff45154b69bac53397bb9ce84c_SeqId_28_ 2023-03-23 08:56:01,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__c8nderk1phs0el1ldm1vqfgu9jebf1ikk2t99k28267c7tg4855dfdrnlq3n2r5a/family1/e048604627d14e3795dad104af37306f into 6ce47bdacf42565a37db21d6312c1b4c/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/9086c2ff45154b69bac53397bb9ce84c_SeqId_28_ - updating store file list. 2023-03-23 08:56:01,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/9086c2ff45154b69bac53397bb9ce84c_SeqId_28_ into 6ce47bdacf42565a37db21d6312c1b4c/family1 2023-03-23 08:56:01,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__c8nderk1phs0el1ldm1vqfgu9jebf1ikk2t99k28267c7tg4855dfdrnlq3n2r5a/family1/e048604627d14e3795dad104af37306f into 6ce47bdacf42565a37db21d6312c1b4c/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/family1/9086c2ff45154b69bac53397bb9ce84c_SeqId_28_) 2023-03-23 08:56:01,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__c8nderk1phs0el1ldm1vqfgu9jebf1ikk2t99k28267c7tg4855dfdrnlq3n2r5a/family1/e048604627d14e3795dad104af37306f 2023-03-23 08:56:01,751 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:01,751 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:01,753 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032339cb42d623e7468d8b41f580ad007b86 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032339cb42d623e7468d8b41f580ad007b86 2023-03-23 08:56:01,753 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a58719334964443ebdb7dd16006e9476 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a58719334964443ebdb7dd16006e9476 2023-03-23 08:56:01,754 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a9bc1c869bdf4687b6b4d860a5d2c154 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323a9bc1c869bdf4687b6b4d860a5d2c154 2023-03-23 08:56:01,755 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323bc757c9955bd47f19bdfa808f0d57426 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230323bc757c9955bd47f19bdfa808f0d57426 2023-03-23 08:56:01,755 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-23 08:56:01,755 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230323' 2023-03-23 08:56:01,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:01,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:02,299 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303237ec55776bad24a0ba99899dae5d667e8 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303237ec55776bad24a0ba99899dae5d667e8 2023-03-23 08:56:02,300 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:02,339 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230323/family1/f39d351f7f4d42ceb2a53604ab668ce1 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-23 08:56:02,340 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230323/family1/f39d351f7f4d42ceb2a53604ab668ce1}] 2023-03-23 08:56:02,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230323/family1/f39d351f7f4d42ceb2a53604ab668ce1 for inclusion in ab62bf5a7404efdd4302415b58f0194b/family1 2023-03-23 08:56:02,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-23 08:56:02,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-23 08:56:02,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:56:02,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230323/family1/f39d351f7f4d42ceb2a53604ab668ce1 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__s7830e0i2v055m7bq541b7rv29r61nss87i6rl9apo7095pikurt9jbj2551vng2/family1/f39d351f7f4d42ceb2a53604ab668ce1 2023-03-23 08:56:02,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__s7830e0i2v055m7bq541b7rv29r61nss87i6rl9apo7095pikurt9jbj2551vng2/family1/f39d351f7f4d42ceb2a53604ab668ce1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2b45d968be04f269df4621824845c88_SeqId_28_ 2023-03-23 08:56:02,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__s7830e0i2v055m7bq541b7rv29r61nss87i6rl9apo7095pikurt9jbj2551vng2/family1/f39d351f7f4d42ceb2a53604ab668ce1 into ab62bf5a7404efdd4302415b58f0194b/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2b45d968be04f269df4621824845c88_SeqId_28_ - updating store file list. 2023-03-23 08:56:02,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2b45d968be04f269df4621824845c88_SeqId_28_ into ab62bf5a7404efdd4302415b58f0194b/family1 2023-03-23 08:56:02,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__s7830e0i2v055m7bq541b7rv29r61nss87i6rl9apo7095pikurt9jbj2551vng2/family1/f39d351f7f4d42ceb2a53604ab668ce1 into ab62bf5a7404efdd4302415b58f0194b/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/family1/b2b45d968be04f269df4621824845c88_SeqId_28_) 2023-03-23 08:56:02,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__s7830e0i2v055m7bq541b7rv29r61nss87i6rl9apo7095pikurt9jbj2551vng2/family1/f39d351f7f4d42ceb2a53604ab668ce1 2023-03-23 08:56:02,398 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:02,399 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:02,400 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303231954a49c8dee4faab8721961bc5680ed to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303231954a49c8dee4faab8721961bc5680ed 2023-03-23 08:56:02,401 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303233b6a49790aea4f828d4681a04c83c509 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303233b6a49790aea4f828d4681a04c83c509 2023-03-23 08:56:02,402 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323e9b4a1c5bc694b78914a345e8d652483 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323e9b4a1c5bc694b78914a345e8d652483 2023-03-23 08:56:02,403 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323ea61ac89b2b74ccaa1e4d460dfabed81 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323ea61ac89b2b74ccaa1e4d460dfabed81 2023-03-23 08:56:02,403 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-23 08:56:02,403 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230323' 2023-03-23 08:56:02,403 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:02,403 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:03,343 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230323a3438589b53a4622a14234dc01c8d5a0 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a3438589b53a4622a14234dc01c8d5a0 2023-03-23 08:56:03,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:03,384 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230323/family1/36f5b89525374ea7ab9ea4f0e5213706 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-23 08:56:03,385 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04., hostname=jenkins-hbase4.apache.org,32923,1679561616941, seqNum=2 for row with hfile group [{family1,hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230323/family1/36f5b89525374ea7ab9ea4f0e5213706}] 2023-03-23 08:56:03,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(763): Validating hfile at hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230323/family1/36f5b89525374ea7ab9ea4f0e5213706 for inclusion in 1f5319a61c93e4939d4243ead872bb04/family1 2023-03-23 08:56:03,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-23 08:56:03,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(777): Region bounds: first= last=1 2023-03-23 08:56:03,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegion(2446): Flush status journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:56:03,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230323/family1/36f5b89525374ea7ab9ea4f0e5213706 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__1brh1kn19i5kh0m3a7s6buqjbu35ftr2ppgtd3nit1tdllm35h5t4kmp2a2qog4m/family1/36f5b89525374ea7ab9ea4f0e5213706 2023-03-23 08:56:03,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__1brh1kn19i5kh0m3a7s6buqjbu35ftr2ppgtd3nit1tdllm35h5t4kmp2a2qog4m/family1/36f5b89525374ea7ab9ea4f0e5213706 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/1c25c85ea7f94279bee03c539b800ad8_SeqId_28_ 2023-03-23 08:56:03,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(849): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__1brh1kn19i5kh0m3a7s6buqjbu35ftr2ppgtd3nit1tdllm35h5t4kmp2a2qog4m/family1/36f5b89525374ea7ab9ea4f0e5213706 into 1f5319a61c93e4939d4243ead872bb04/family1 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/1c25c85ea7f94279bee03c539b800ad8_SeqId_28_ - updating store file list. 2023-03-23 08:56:03,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(882): Loaded HFile hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/1c25c85ea7f94279bee03c539b800ad8_SeqId_28_ into 1f5319a61c93e4939d4243ead872bb04/family1 2023-03-23 08:56:03,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.HStore(855): Successfully loaded hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__1brh1kn19i5kh0m3a7s6buqjbu35ftr2ppgtd3nit1tdllm35h5t4kmp2a2qog4m/family1/36f5b89525374ea7ab9ea4f0e5213706 into 1f5319a61c93e4939d4243ead872bb04/family1 (new location: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/family1/1c25c85ea7f94279bee03c539b800ad8_SeqId_28_) 2023-03-23 08:56:03,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=32923] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/staging/jenkins__testMajorCompactionFromAdmin__1brh1kn19i5kh0m3a7s6buqjbu35ftr2ppgtd3nit1tdllm35h5t4kmp2a2qog4m/family1/36f5b89525374ea7ab9ea4f0e5213706 2023-03-23 08:56:03,432 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230323 table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:03,433 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:03,434 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032335a758f654d846da8e3ee5a665525a56 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032335a758f654d846da8e3ee5a665525a56 2023-03-23 08:56:03,435 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303234c5ebefcf4974fe9afb36353b95f9400 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303234c5ebefcf4974fe9afb36353b95f9400 2023-03-23 08:56:03,436 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a3e59710ee364178aed777b231237102 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a3e59710ee364178aed777b231237102 2023-03-23 08:56:03,437 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a77ad661ffcf4ddb90a5ee5e9289d293 to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230323a77ad661ffcf4ddb90a5ee5e9289d293 2023-03-23 08:56:03,437 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-23 08:56:03,437 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230323' 2023-03-23 08:56:03,437 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230323 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:03,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-23 08:56:03,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-23 08:56:03,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del] 2023-03-23 08:56:03,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:03,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032317c1bf83a05c481d88d15d79ae086d1d_del 2023-03-23 08:56:03,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-23 08:56:03,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del] 2023-03-23 08:56:03,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:03,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032371f9a3282cb54974aaf75977ae0cb387_del 2023-03-23 08:56:03,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-23 08:56:03,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] compactions.PartitionedMobCompactor(367): [hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del] 2023-03-23 08:56:03,440 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(360): Archiving compacted files. 2023-03-23 08:56:03,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del to hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230323eeda015b579d453c994d099e199cae02_del 2023-03-23 08:56:03,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-23 08:56:03,442 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679561621037] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-23 08:56:03,443 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-23 08:56:03,444 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.1100 sec 2023-03-23 08:56:03,460 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:56:03,461 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:56:03,464 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-23 08:56:03,464 DEBUG [Listener at localhost/38371] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-23 08:56:03,468 INFO [Listener at localhost/38371] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-23 08:56:03,503 INFO [Listener at localhost/38371] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 261), OpenFileDescriptor=633 (was 659), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=303 (was 340), ProcessCount=169 (was 169), AvailableMemoryMB=5506 (was 5520) 2023-03-23 08:56:03,503 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b944ad to 127.0.0.1:50031 2023-03-23 08:56:03,503 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:03,504 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-23 08:56:03,504 INFO [Listener at localhost/38371] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:56:03,504 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c1e7f16 to 127.0.0.1:50031 2023-03-23 08:56:03,504 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:03,504 DEBUG [Listener at localhost/38371] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-23 08:56:03,504 DEBUG [Listener at localhost/38371] util.JVMClusterUtil(257): Found active master hash=1763081285, stopped=false 2023-03-23 08:56:03,504 INFO [Listener at localhost/38371] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:56:03,506 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-23 08:56:03,506 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-23 08:56:03,506 INFO [Listener at localhost/38371] procedure2.ProcedureExecutor(629): Stopping 2023-03-23 08:56:03,506 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:56:03,506 DEBUG [Listener at localhost/38371] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16dda0de to 127.0.0.1:50031 2023-03-23 08:56:03,506 DEBUG [Listener at localhost/38371] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:03,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-23 08:56:03,506 INFO [Listener at localhost/38371] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,32923,1679561616941' ***** 2023-03-23 08:56:03,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-23 08:56:03,506 INFO [Listener at localhost/38371] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-23 08:56:03,507 INFO [RS:0;jenkins-hbase4:32923] regionserver.HeapMemoryManager(220): Stopping 2023-03-23 08:56:03,507 INFO [RS:0;jenkins-hbase4:32923] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-23 08:56:03,507 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-23 08:56:03,507 INFO [RS:0;jenkins-hbase4:32923] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-23 08:56:03,507 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 8316b0845ad8dc2f32d05f84414a28c1 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 1f5319a61c93e4939d4243ead872bb04 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 80f02bad9c2288060a6ec1f940d1637b 2023-03-23 08:56:03,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8316b0845ad8dc2f32d05f84414a28c1, disabling compactions & flushes 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for bdaddcf1cd39563f63add7141876dbbe 2023-03-23 08:56:03,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 2956fdc6f28aecbd88e7bf44df609d8b 2023-03-23 08:56:03,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for be2d00f2bc48fa70033bcc38f2ad948a 2023-03-23 08:56:03,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. after waiting 0 ms 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 6469db6845c8da10a08493a71e41b196 2023-03-23 08:56:03,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for ab62bf5a7404efdd4302415b58f0194b 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 6ce47bdacf42565a37db21d6312c1b4c 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for b39222917b68c0e24683b30b78e35130 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 1da1b8621f257956854515270c9db5ce 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for d69f3c8983e2fde2ae01956e233272af 2023-03-23 08:56:03,508 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 1735e7d5d07578998f62eda41d807aa2 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 2b2e5173dbc57f26ba388a87c96e3113 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 94e66683fc31732d625f937ede51eb76 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 6567fac53213b6f4d692c06a6cbc7ae8 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for d2a2668ed83ec24163021b10f82d1b3e 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for caf47a88030410666d2a8cb9e80797cb 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:56:03,509 DEBUG [RS:0;jenkins-hbase4:32923] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66db461f to 127.0.0.1:50031 2023-03-23 08:56:03,509 DEBUG [RS:0;jenkins-hbase4:32923] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-23 08:56:03,509 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-23 08:56:03,510 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-23 08:56:03,510 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-23 08:56:03,510 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-23 08:56:03,510 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-23 08:56:03,510 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-23 08:56:03,510 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-23 08:56:03,510 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-23 08:56:03,510 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740, 1735e7d5d07578998f62eda41d807aa2, 1da1b8621f257956854515270c9db5ce, 1f5319a61c93e4939d4243ead872bb04, 2956fdc6f28aecbd88e7bf44df609d8b, 2b2e5173dbc57f26ba388a87c96e3113, 6469db6845c8da10a08493a71e41b196, 6567fac53213b6f4d692c06a6cbc7ae8, 6ce47bdacf42565a37db21d6312c1b4c, 80f02bad9c2288060a6ec1f940d1637b, 8316b0845ad8dc2f32d05f84414a28c1, 94e66683fc31732d625f937ede51eb76, ab62bf5a7404efdd4302415b58f0194b, b39222917b68c0e24683b30b78e35130, bdaddcf1cd39563f63add7141876dbbe, be2d00f2bc48fa70033bcc38f2ad948a, caf47a88030410666d2a8cb9e80797cb, d2a2668ed83ec24163021b10f82d1b3e, d69f3c8983e2fde2ae01956e233272af, df4c4c6f1021c2dacf2331fe9a27397f, fadaad8dfb5b4d4096ebaa9bf3fe536f 2023-03-23 08:56:03,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/8316b0845ad8dc2f32d05f84414a28c1/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:56:03,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8316b0845ad8dc2f32d05f84414a28c1: 2023-03-23 08:56:03,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1679561721348.8316b0845ad8dc2f32d05f84414a28c1. 2023-03-23 08:56:03,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1f5319a61c93e4939d4243ead872bb04, disabling compactions & flushes 2023-03-23 08:56:03,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:56:03,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:56:03,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. after waiting 0 ms 2023-03-23 08:56:03,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:56:03,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/1f5319a61c93e4939d4243ead872bb04/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:56:03,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1f5319a61c93e4939d4243ead872bb04: 2023-03-23 08:56:03,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1679561745391.1f5319a61c93e4939d4243ead872bb04. 2023-03-23 08:56:03,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 80f02bad9c2288060a6ec1f940d1637b, disabling compactions & flushes 2023-03-23 08:56:03,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:56:03,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:56:03,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. after waiting 0 ms 2023-03-23 08:56:03,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:56:03,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyPolicy/80f02bad9c2288060a6ec1f940d1637b/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-23 08:56:03,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 80f02bad9c2288060a6ec1f940d1637b: 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1679561657999.80f02bad9c2288060a6ec1f940d1637b. 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bdaddcf1cd39563f63add7141876dbbe, disabling compactions & flushes 2023-03-23 08:56:03,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. after waiting 0 ms 2023-03-23 08:56:03,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:56:03,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithMonthlyPolicy/bdaddcf1cd39563f63add7141876dbbe/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-23 08:56:03,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:56:03,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bdaddcf1cd39563f63add7141876dbbe: 2023-03-23 08:56:03,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1679561633872.bdaddcf1cd39563f63add7141876dbbe. 2023-03-23 08:56:03,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2956fdc6f28aecbd88e7bf44df609d8b, disabling compactions & flushes 2023-03-23 08:56:03,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:56:03,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:56:03,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. after waiting 0 ms 2023-03-23 08:56:03,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:56:03,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithMonthlyPolicy/2956fdc6f28aecbd88e7bf44df609d8b/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-23 08:56:03,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2956fdc6f28aecbd88e7bf44df609d8b: 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1679561737743.2956fdc6f28aecbd88e7bf44df609d8b. 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing be2d00f2bc48fa70033bcc38f2ad948a, disabling compactions & flushes 2023-03-23 08:56:03,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. after waiting 0 ms 2023-03-23 08:56:03,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:56:03,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/be2d00f2bc48fa70033bcc38f2ad948a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-23 08:56:03,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:56:03,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for be2d00f2bc48fa70033bcc38f2ad948a: 2023-03-23 08:56:03,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1679561654529.be2d00f2bc48fa70033bcc38f2ad948a. 2023-03-23 08:56:03,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6469db6845c8da10a08493a71e41b196, disabling compactions & flushes 2023-03-23 08:56:03,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:56:03,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:56:03,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. after waiting 0 ms 2023-03-23 08:56:03,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:56:03,538 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-23 08:56:03,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/6469db6845c8da10a08493a71e41b196/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-23 08:56:03,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6469db6845c8da10a08493a71e41b196: 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1679561697662.6469db6845c8da10a08493a71e41b196. 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ab62bf5a7404efdd4302415b58f0194b, disabling compactions & flushes 2023-03-23 08:56:03,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. after waiting 0 ms 2023-03-23 08:56:03,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:56:03,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/ab62bf5a7404efdd4302415b58f0194b/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ab62bf5a7404efdd4302415b58f0194b: 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1679561745391.ab62bf5a7404efdd4302415b58f0194b. 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6ce47bdacf42565a37db21d6312c1b4c, disabling compactions & flushes 2023-03-23 08:56:03,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. after waiting 0 ms 2023-03-23 08:56:03,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:56:03,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionFromAdmin/6ce47bdacf42565a37db21d6312c1b4c/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:56:03,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6ce47bdacf42565a37db21d6312c1b4c: 2023-03-23 08:56:03,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1679561745391.6ce47bdacf42565a37db21d6312c1b4c. 2023-03-23 08:56:03,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b39222917b68c0e24683b30b78e35130, disabling compactions & flushes 2023-03-23 08:56:03,552 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:56:03,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:56:03,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. after waiting 0 ms 2023-03-23 08:56:03,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:56:03,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/b39222917b68c0e24683b30b78e35130/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-23 08:56:03,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:56:03,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b39222917b68c0e24683b30b78e35130: 2023-03-23 08:56:03,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1679561654529.b39222917b68c0e24683b30b78e35130. 2023-03-23 08:56:03,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1da1b8621f257956854515270c9db5ce, disabling compactions & flushes 2023-03-23 08:56:03,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:56:03,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:56:03,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. after waiting 0 ms 2023-03-23 08:56:03,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:56:03,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/1da1b8621f257956854515270c9db5ce/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-23 08:56:03,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1da1b8621f257956854515270c9db5ce: 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1679561697662.1da1b8621f257956854515270c9db5ce. 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d69f3c8983e2fde2ae01956e233272af, disabling compactions & flushes 2023-03-23 08:56:03,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. after waiting 0 ms 2023-03-23 08:56:03,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:56:03,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d69f3c8983e2fde2ae01956e233272af/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d69f3c8983e2fde2ae01956e233272af: 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1679561721348.d69f3c8983e2fde2ae01956e233272af. 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fadaad8dfb5b4d4096ebaa9bf3fe536f, disabling compactions & flushes 2023-03-23 08:56:03,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. after waiting 0 ms 2023-03-23 08:56:03,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:56:03,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/fadaad8dfb5b4d4096ebaa9bf3fe536f/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-23 08:56:03,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fadaad8dfb5b4d4096ebaa9bf3fe536f: 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679561679613.fadaad8dfb5b4d4096ebaa9bf3fe536f. 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing df4c4c6f1021c2dacf2331fe9a27397f, disabling compactions & flushes 2023-03-23 08:56:03,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. after waiting 0 ms 2023-03-23 08:56:03,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:56:03,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing df4c4c6f1021c2dacf2331fe9a27397f 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-23 08:56:03,577 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-23 08:56:03,628 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-23 08:56:03,628 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-23 08:56:03,711 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740, 1735e7d5d07578998f62eda41d807aa2, 2b2e5173dbc57f26ba388a87c96e3113, 6567fac53213b6f4d692c06a6cbc7ae8, 94e66683fc31732d625f937ede51eb76, caf47a88030410666d2a8cb9e80797cb, d2a2668ed83ec24163021b10f82d1b3e, df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:56:03,911 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740, 1735e7d5d07578998f62eda41d807aa2, 2b2e5173dbc57f26ba388a87c96e3113, 6567fac53213b6f4d692c06a6cbc7ae8, 94e66683fc31732d625f937ede51eb76, caf47a88030410666d2a8cb9e80797cb, d2a2668ed83ec24163021b10f82d1b3e, df4c4c6f1021c2dacf2331fe9a27397f 2023-03-23 08:56:03,925 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/info/099cbdebe5f64766bb30696c500c0b6c 2023-03-23 08:56:03,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 099cbdebe5f64766bb30696c500c0b6c 2023-03-23 08:56:03,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/.tmp/info/11e327c039934603b585e45b8cd675cd 2023-03-23 08:56:03,975 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/.tmp/info/11e327c039934603b585e45b8cd675cd as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/info/11e327c039934603b585e45b8cd675cd 2023-03-23 08:56:03,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/info/11e327c039934603b585e45b8cd675cd, entries=3, sequenceid=7, filesize=4.8 K 2023-03-23 08:56:03,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for df4c4c6f1021c2dacf2331fe9a27397f in 414ms, sequenceid=7, compaction requested=false 2023-03-23 08:56:03,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/namespace/df4c4c6f1021c2dacf2331fe9a27397f/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-23 08:56:03,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for df4c4c6f1021c2dacf2331fe9a27397f: 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1679561620015.df4c4c6f1021c2dacf2331fe9a27397f. 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1735e7d5d07578998f62eda41d807aa2, disabling compactions & flushes 2023-03-23 08:56:03,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. after waiting 0 ms 2023-03-23 08:56:03,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/1735e7d5d07578998f62eda41d807aa2/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-23 08:56:03,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1735e7d5d07578998f62eda41d807aa2: 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679561621188.1735e7d5d07578998f62eda41d807aa2. 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2b2e5173dbc57f26ba388a87c96e3113, disabling compactions & flushes 2023-03-23 08:56:03,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. after waiting 0 ms 2023-03-23 08:56:03,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:56:03,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testScannerOnBulkLoadRefHFiles/2b2e5173dbc57f26ba388a87c96e3113/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-23 08:56:03,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:56:03,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2b2e5173dbc57f26ba388a87c96e3113: 2023-03-23 08:56:03,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1679561654529.2b2e5173dbc57f26ba388a87c96e3113. 2023-03-23 08:56:03,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 94e66683fc31732d625f937ede51eb76, disabling compactions & flushes 2023-03-23 08:56:03,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:56:03,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:56:03,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. after waiting 0 ms 2023-03-23 08:56:03,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:56:03,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMinorCompactionWithWeeklyPolicy/94e66683fc31732d625f937ede51eb76/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-23 08:56:03,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 94e66683fc31732d625f937ede51eb76: 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1679561641349.94e66683fc31732d625f937ede51eb76. 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6567fac53213b6f4d692c06a6cbc7ae8, disabling compactions & flushes 2023-03-23 08:56:03,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. after waiting 0 ms 2023-03-23 08:56:03,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:56:03,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/6567fac53213b6f4d692c06a6cbc7ae8/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-23 08:56:03,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:56:03,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6567fac53213b6f4d692c06a6cbc7ae8: 2023-03-23 08:56:03,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679561666973.6567fac53213b6f4d692c06a6cbc7ae8. 2023-03-23 08:56:03,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2a2668ed83ec24163021b10f82d1b3e, disabling compactions & flushes 2023-03-23 08:56:03,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:56:03,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:56:03,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. after waiting 0 ms 2023-03-23 08:56:03,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:56:03,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/ns/testMinorCompaction/d2a2668ed83ec24163021b10f82d1b3e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-23 08:56:03,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:56:03,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2a2668ed83ec24163021b10f82d1b3e: 2023-03-23 08:56:03,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1679561721348.d2a2668ed83ec24163021b10f82d1b3e. 2023-03-23 08:56:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing caf47a88030410666d2a8cb9e80797cb, disabling compactions & flushes 2023-03-23 08:56:03,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:56:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:56:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. after waiting 0 ms 2023-03-23 08:56:03,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:56:04,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/default/testCompactionWithHFileLink/caf47a88030410666d2a8cb9e80797cb/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-23 08:56:04,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:56:04,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for caf47a88030410666d2a8cb9e80797cb: 2023-03-23 08:56:04,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1679561697662.caf47a88030410666d2a8cb9e80797cb. 2023-03-23 08:56:04,111 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-23 08:56:04,312 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-23 08:56:04,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/rep_barrier/bf991d968edb4e1d80661a223a1d2a7f 2023-03-23 08:56:04,337 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf991d968edb4e1d80661a223a1d2a7f 2023-03-23 08:56:04,512 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-03-23 08:56:04,512 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-03-23 08:56:04,512 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-23 08:56:04,712 DEBUG [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-23 08:56:04,744 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/table/3edc25afe7d24cad9c77ab7e584b11cb 2023-03-23 08:56:04,746 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3edc25afe7d24cad9c77ab7e584b11cb 2023-03-23 08:56:04,747 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/info/099cbdebe5f64766bb30696c500c0b6c as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/info/099cbdebe5f64766bb30696c500c0b6c 2023-03-23 08:56:04,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 099cbdebe5f64766bb30696c500c0b6c 2023-03-23 08:56:04,750 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/info/099cbdebe5f64766bb30696c500c0b6c, entries=226, sequenceid=130, filesize=37.9 K 2023-03-23 08:56:04,750 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/rep_barrier/bf991d968edb4e1d80661a223a1d2a7f as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/rep_barrier/bf991d968edb4e1d80661a223a1d2a7f 2023-03-23 08:56:04,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf991d968edb4e1d80661a223a1d2a7f 2023-03-23 08:56:04,754 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/rep_barrier/bf991d968edb4e1d80661a223a1d2a7f, entries=1, sequenceid=130, filesize=5.0 K 2023-03-23 08:56:04,755 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/.tmp/table/3edc25afe7d24cad9c77ab7e584b11cb as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/table/3edc25afe7d24cad9c77ab7e584b11cb 2023-03-23 08:56:04,758 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3edc25afe7d24cad9c77ab7e584b11cb 2023-03-23 08:56:04,758 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/table/3edc25afe7d24cad9c77ab7e584b11cb, entries=27, sequenceid=130, filesize=6.6 K 2023-03-23 08:56:04,760 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1250ms, sequenceid=130, compaction requested=false 2023-03-23 08:56:04,763 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-23 08:56:04,764 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-23 08:56:04,764 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-23 08:56:04,764 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-23 08:56:04,764 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-23 08:56:04,912 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,32923,1679561616941; all regions closed. 2023-03-23 08:56:04,919 DEBUG [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/oldWALs 2023-03-23 08:56:04,919 INFO [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C32923%2C1679561616941.meta:.meta(num 1679561619757) 2023-03-23 08:56:04,922 DEBUG [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/oldWALs 2023-03-23 08:56:04,923 INFO [RS:0;jenkins-hbase4:32923] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C32923%2C1679561616941:(num 1679561618776) 2023-03-23 08:56:04,923 DEBUG [RS:0;jenkins-hbase4:32923] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:04,923 INFO [RS:0;jenkins-hbase4:32923] regionserver.LeaseManager(133): Closed leases 2023-03-23 08:56:04,923 INFO [RS:0;jenkins-hbase4:32923] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-23 08:56:04,923 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-23 08:56:04,924 INFO [RS:0;jenkins-hbase4:32923] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:32923 2023-03-23 08:56:04,933 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-23 08:56:04,933 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,32923,1679561616941 2023-03-23 08:56:04,934 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@31ed7261 rejected from java.util.concurrent.ThreadPoolExecutor@35d2f084[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:04,934 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-23 08:56:04,934 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1bceb88f rejected from java.util.concurrent.ThreadPoolExecutor@35d2f084[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:04,934 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,32923,1679561616941] 2023-03-23 08:56:04,934 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,32923,1679561616941; numProcessing=1 2023-03-23 08:56:04,936 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,32923,1679561616941 already deleted, retry=false 2023-03-23 08:56:04,936 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,32923,1679561616941 expired; onlineServers=0 2023-03-23 08:56:04,936 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37881,1679561615669' ***** 2023-03-23 08:56:04,936 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-23 08:56:04,936 DEBUG [M:0;jenkins-hbase4:37881] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@282b767e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-23 08:56:04,936 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37881,1679561615669 2023-03-23 08:56:04,936 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37881,1679561615669; all regions closed. 2023-03-23 08:56:04,936 DEBUG [M:0;jenkins-hbase4:37881] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:04,936 DEBUG [M:0;jenkins-hbase4:37881] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-23 08:56:04,937 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-23 08:56:04,937 DEBUG [M:0;jenkins-hbase4:37881] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-23 08:56:04,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679561618488] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679561618488,5,FailOnTimeoutGroup] 2023-03-23 08:56:04,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679561618489] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679561618489,5,FailOnTimeoutGroup] 2023-03-23 08:56:04,937 INFO [M:0;jenkins-hbase4:37881] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-23 08:56:04,937 INFO [M:0;jenkins-hbase4:37881] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-23 08:56:04,937 INFO [M:0;jenkins-hbase4:37881] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-23 08:56:04,937 DEBUG [M:0;jenkins-hbase4:37881] master.HMaster(1512): Stopping service threads 2023-03-23 08:56:04,938 INFO [M:0;jenkins-hbase4:37881] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-23 08:56:04,938 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-23 08:56:04,938 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-23 08:56:04,938 ERROR [M:0;jenkins-hbase4:37881] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-23 08:56:04,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-23 08:56:04,938 INFO [M:0;jenkins-hbase4:37881] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-23 08:56:04,939 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-23 08:56:04,939 DEBUG [M:0;jenkins-hbase4:37881] zookeeper.ZKUtil(398): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-23 08:56:04,939 WARN [M:0;jenkins-hbase4:37881] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-23 08:56:04,939 INFO [M:0;jenkins-hbase4:37881] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-23 08:56:04,939 INFO [M:0;jenkins-hbase4:37881] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-23 08:56:04,939 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-23 08:56:04,939 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:56:04,939 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:56:04,939 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-23 08:56:04,939 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:56:04,939 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.46 KB heapSize=598.08 KB 2023-03-23 08:56:05,035 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-23 08:56:05,035 INFO [RS:0;jenkins-hbase4:32923] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,32923,1679561616941; zookeeper connection closed. 2023-03-23 08:56:05,035 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@631e3184 rejected from java.util.concurrent.ThreadPoolExecutor@35d2f084[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:05,035 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): regionserver:32923-0x10110944f9e0001, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-23 08:56:05,035 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5e5acb12] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5e5acb12 2023-03-23 08:56:05,035 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7e1b5f45 rejected from java.util.concurrent.ThreadPoolExecutor@35d2f084[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:05,036 INFO [Listener at localhost/38371] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-23 08:56:05,347 INFO [M:0;jenkins-hbase4:37881] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.46 KB at sequenceid=1004 (bloomFilter=true), to=hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/cafc77e19c544b4ab30bc9ec54c28cd3 2023-03-23 08:56:05,350 INFO [M:0;jenkins-hbase4:37881] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cafc77e19c544b4ab30bc9ec54c28cd3 2023-03-23 08:56:05,351 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/cafc77e19c544b4ab30bc9ec54c28cd3 as hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/cafc77e19c544b4ab30bc9ec54c28cd3 2023-03-23 08:56:05,354 INFO [M:0;jenkins-hbase4:37881] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cafc77e19c544b4ab30bc9ec54c28cd3 2023-03-23 08:56:05,354 INFO [M:0;jenkins-hbase4:37881] regionserver.HStore(1080): Added hdfs://localhost:42513/user/jenkins/test-data/2bd038e7-96b8-4fc1-6e46-591fbbf68376/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/cafc77e19c544b4ab30bc9ec54c28cd3, entries=123, sequenceid=1004, filesize=33.3 K 2023-03-23 08:56:05,354 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegion(2948): Finished flush of dataSize ~512.46 KB/524762, heapSize ~598.06 KB/612416, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1004, compaction requested=false 2023-03-23 08:56:05,355 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-23 08:56:05,355 DEBUG [M:0;jenkins-hbase4:37881] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-23 08:56:05,358 INFO [M:0;jenkins-hbase4:37881] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-23 08:56:05,358 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-23 08:56:05,358 INFO [M:0;jenkins-hbase4:37881] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37881 2023-03-23 08:56:05,361 DEBUG [M:0;jenkins-hbase4:37881] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37881,1679561615669 already deleted, retry=false 2023-03-23 08:56:05,462 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-23 08:56:05,462 INFO [M:0;jenkins-hbase4:37881] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37881,1679561615669; zookeeper connection closed. 2023-03-23 08:56:05,462 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@35053703 rejected from java.util.concurrent.ThreadPoolExecutor@384dad61[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 903] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:05,463 DEBUG [Listener at localhost/38371-EventThread] zookeeper.ZKWatcher(600): master:37881-0x10110944f9e0000, quorum=127.0.0.1:50031, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-23 08:56:05,463 ERROR [Listener at localhost/38371-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a1bf6f0 rejected from java.util.concurrent.ThreadPoolExecutor@384dad61[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 903] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-23 08:56:05,464 WARN [Listener at localhost/38371] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-23 08:56:05,468 INFO [Listener at localhost/38371] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-23 08:56:05,592 WARN [BP-454665439-172.31.14.131-1679561612282 heartbeating to localhost/127.0.0.1:42513] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-23 08:56:05,592 WARN [BP-454665439-172.31.14.131-1679561612282 heartbeating to localhost/127.0.0.1:42513] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-454665439-172.31.14.131-1679561612282 (Datanode Uuid 53bda90c-65e6-4972-9c12-411d3cabc1c9) service to localhost/127.0.0.1:42513 2023-03-23 08:56:05,592 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-23 08:56:05,592 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ea8dd42 to 127.0.0.1:50031 2023-03-23 08:56:05,592 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-23 08:56:05,594 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/dfs/data/data1/current/BP-454665439-172.31.14.131-1679561612282] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-23 08:56:05,595 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/dc5138f3-1cf0-95a0-ef3c-5cc0c1b30071/cluster_e052df60-640c-2150-f6a4-dbdef39999f2/dfs/data/data2/current/BP-454665439-172.31.14.131-1679561612282] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-23 08:56:05,619 INFO [Listener at localhost/38371] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-23 08:56:05,750 INFO [Listener at localhost/38371] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-23 08:56:05,841 INFO [Listener at localhost/38371] hbase.HBaseTestingUtility(1293): Minicluster is down 2023-03-23 08:56:05,844 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint